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The Wire-Tap Channel 


By A. D. WYNER 
(Manuscript received May 9, 1975) 


We consider the situation in which digital data 1s to be reliably trans- 
mitted over a discrete, memoryless channel (DMC) that is subjected to a 
wire-tap at the receiver. We assume that the wire-tapper views the channel 
output via a second DMC. Encoding by the transmitter and decoding by the 
receiver are permitted. However, the code books used tn these operations are 
assumed to be known by the wire-tapper. The designer attempts to build 
the encoder-decoder in such a way as to maximize the transmission rate R, 
and the equivocation d of the data as seen by the wire-tapper. In this paper, 
we find the trade-off curve between R and d, assuming essentially perfect 
(“error-free”) transmission. In particular, if d is equal to Hs, the entropy 
of the data source, then we consider that the transmission is accomplished 
an perfect secrecy. Our results imply that there exists a C, > 0, such 
that reliable transmission at rates up to C’; is possible in approximately 
perfect secrecy. 


I]. INTRODUCTION 


In this paper we study a (perhaps noisy) communication system 
that is being wire-tapped via a second noisy channel. Our object is to 
encode the data in such a way that the wire-tapper’s level of confusion 
will be as high as possible. To fix ideas, consider first the simple special 
case depicted in Fig. 1 (in which the main communication system is 
noiseless). The source emits a data sequence Sj, Se, ---, which consists 
of independent copies of the binary random variable S, where 
Pr {S = 0} = Pr {S = 1} = 4. The encoder examines the first K 
source bits S* = (Si, ---, Sx) and encodes S* into a binary N vector 
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Fig. 1—Wire-tap channel (special case). 


= (X,, ---, Xy). X* in turn is transmitted perfectly to the decoder 
via at seeiee channel and is transformed into a binary data stream 
Sx = ($1, -++, Sx) for delivery to the destination. The “error proba- 
bility”’ is dehued as 


K i 
zB r {S, A Sx}. (1) 


an 


The entire process is repeated on successive blocks of K source bits. 
The transmission rate is K/N bits per transmitted channel symbol. 

The wire-tapper observes the encoded vector X” through a (memory- 
less) binary symmetric channel (Bsc) with crossover probability 
po(O < po S $). The corresponding output at the wire-tap is Z% 
= (Z1, :--, Zv), so that forz,z =0,101SnSQN), 


Pr {Zn = ra = x} = (1 — Do) 6x,2 + po(l — 52,2). 


We take the equivocation 


eae x HIS |Z") (2) 


as a measure of the degree to which the wire-tapper is confused. The 
logarithms in H are, as are all logarithms in this paper, taken to the 
base 2. The system designer would like to have P, close to zero, with 
K/N and A as large as possible. 

Consider the following schemes: 


(1) Set K = N =1, and let X; =S8,. This results in P, = 0, 
K/N = 1, and A = H(X1|Z1) = h(po), where 


h(A) = — rAlogA — (1 — A) log (1 — A), OSA S1, (3) 
(take 0 log 0 = 0). 
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(zt) Set K = 1, and let N be arbitrary. Let Co be the subset of 
binary N space, {0, 1}, consisting of those N vectors with even parity 
(i.e., an even number of 1’s). Let C; € {0, 1}% be the subset of vectors 
with odd parity. The encoder works as follows. When S; = 7, (¢ = 0, 1), 
the encoder output X” is a randomly chosen vector in C;. Thus, the 
encoder is a channel with transition probability 


2-A-)D, x EC, 
0, x & C;, 


for 7 = 0, 1. Clearly, the decoder can recover S; from X*% perfectly, so 
that P, = 0. We now turn to the wire-tapper who observes Z”, the 
output of the Bsc corresponding to the input X”. Let z € {0,1} be 
a vector of, say, even parity. Then 


Pr {X* = x|S, = 7} = | 


Pr {S; = 0|Z* = z} = Pr ee Bsc makes an 


even number of errors 


(NY; N-j= iyi Qn )N 
= du F pall = pa)" = 3 a = 2p)”. 
I= 
j even 


The last equality can be verified by applying the binomial formula to 


EU = pa) + apo = & (% Yoga = po) i e2y 


Then 

2 5 (7) pi = po = = pot 1p)” + = pa = Lp)" 
jeven \ , ey ee 

(S. P. Lloyd). Similarly, for z € {0, 1}* of odd parity, 


the Bsc makes an 
odd number of errors 


=} — 4(1 — 2p.) 


Pr {8:1 = 0|Z" = z} = P| 


Therefore, for all z € {0, 1}¥%, 
H(8,|Z" = z) = ALG — 3(1 — 2po)* ], 


so that 
A = H(S8,|Z¥) = ALg — 3(1 — 2p0)¥] 


—1=H(Si), asN-ow. 


Thus, as N—o, the equivocation at the wire-tap approaches the 
unconditional source entropy, so that communication is accomplished 
in perfect secrecy. The “catch” is that, as N —o, the transmission 
rate K/N = 1/N > 0. 
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A central question to which this paper is addressed is whether or 
not it is possible to transmit at a rate bounded away from zero, and 
yet achieve approximately perfect secrecy, i.e.. A A(S,). Before 
giving the answer to this question, we shall describe the more general 
problem that is addressed in the sequel. 

Refer to Fig. 2. The source is discrete and memoryless with entropy 
Hyg. The ‘main channel” and the “‘wire-tap channel” are discrete 
memoryless channels with transition probabilities Qar(-|-) and 
Qw(-|-), respectively. The source and the transition probabilities Qa 
and Qw are given and fixed. The encoder, as in the above example, is a 
channel with the K vector S* as input and the N vector X* as output. 
The vector X* is in turn the input to the main channel. The main 
channel output and the wire-tap channel input is YY. The wire-tap 
channel output is Z”. The decoder associates a K vector S* with Y¥ ; 
and the error probability P, is given by (1). The equivocation A is 
given by (2), and the transmission rate is KHs/N source bits per 
channel input symbol. Roughly speaking, a pair (R, d) is achievable 
if it is possible to find an encoder-decoder with arbitrarily small P., 
and KHs/N about R, and A about d (with perhaps N and K very 
large). Our main problem is the characterization of the family of 
achievable (R, d) pairs, and such a characterization is given in Theorem 
2. It turns out (Theorem 3) that, in nearly every case, there exists a 
“secrecy capacity,’ C, > 0, such that (C., Hs) is achievable [ while, 
for R > C,, (R, Hs) is not achievable ]. Thus, it is possible to reliably 
transmit information at the positive rate C, in essentially perfect 
secrecy. 

For the special case of our introductory example (Hs = 1, Qu 
corresponding to a noiseless channel and Qy to a Bsc), the conclusion 
of Theorem 2 specializes to the assertion that (R, d) is achievable if 
and onlyifO S$ Rs1,0 $d S1,and Rd S h(p»). Note that scheme 
(t) suggested above for this special case asserts that R = 1, d = h(po) 


MAIN CHANNEL 
WIRE-TAP CHANNEL 
Qw 


ZN 





Fig. 2—Wire-tap channel (general case). 
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is achievable. From Theorem 2, this value of d = h(po) is the maxi- 
mum achievable d, if R = 1. Scheme (77) above asserts that R = 0, 
d = 1 is achievable, but this is distinctly suboptimal since from 
Theorem 2, R = h(po), d = 1 is achievable. Thus, reliable trans- 
mission at a rate h(po) is possible with perfect secrecy, and C, = h(po). 

An outline of the remainder of this paper now follows. In Section 
II, we give a formal statement of the problem and state the main 
results (Theorems 2 and 3). In Section III we give a proof of Theorem 
2 for the special case discussed above (main channel noiseless, wire-tap 
channel a Bsc). In Section IV, we prove the converse half of Theorem 2, 
and in Section V the direct half of that theorem. 


Il. FORMAL STATEMENT OF THE PROBLEM AND SUMMARY OF RESULTS 


In this section we give a precise statement of the problem that we 
stated informally in Section I. We then summarize our results. 

First, a word about notation. Let U be an arbitrary finite set. Denote 
its cardinality by |u|. Consider Ww, the set of N vectors with com- 
ponents in U. The members of U* will be written as 


uv = (uw, U2, °°", un), 


where subscripted letters denote the components and boldface super- 
scripted letters denote vectors. A similar convention applies to random 
vectors and random variables, which are denoted by upper-case letters. 
When the dimension JN of a vector is clear from the context, we omit 
the superscript. 

For random variables X, Y, Z, etc., the notation H(X), H(X|Y), 
I(X;Y), [(X; Y|Z), etc., denotes the standard information quantities 
as defined in Gallager.! The logarithms in these quantities are, as are 
all logarithms in this paper, taken to the base 2. Finally, for n = 3, 4, 
5, -*+, we say that the sequence of random variables {X;}7.; is a 
“Markov chain” if (X1, Xo, ---, Xj-1) and (Xj41, ---, X,) are condi- 
tionally independent, given X;(1 < 7 <n). We make repeated use of 
the fact that, if X1, X2, X3 is a Markov chain, then 


H(X3|X1, Xe) = H(X3|X2). (4) 


At this point we call attention to Appendix A, in which the data- 
processing theorem and Fano’s inequality are given in several forms. 

We now turn to the description of the communication system. We 
assume that the system designer is given a source and two channels 
that are defined as follows. 


(¢) The source is defined by the sequence {S,};°, where the S; are 
independent, identically distributed random variables that take 
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values in the finite set $. We assume that the probability law that 
defines the {S,} is known. Let the entropy H(S;) = Hs. In Appendix 
C we show how to extend the results of this paper to arbitrary station- 
ary finite alphabet ergodic sources. 

(it) The main channel is a discrete memoryless channel with finite 
input alphabet XX, finite output alphabet Y, and transition probability 
Qu(y|z), 2 € XL, y © Y. Since the channel is memoryless, the transi- 
tion probability for N vectors is 


QW (ylx) = IT Qur(yalz). 6) 


Denote the channel capacity of the main channel by Cy. 

(iit) The wire-tap channel is also a discrete memoryless channel 
with input alphabet Y, finite output alphabet 4%, and transition 
probability Qw(z|y), y © Y, 2 € 3. The cascade of the main channel 
and the wire-tap channel is another memoryless channel with transition 
probability 


Quw(z|z) = YL Wwely)Qmu(y|z). (6) 
yey 


Occasionally, when there is no ambiguity, we use the transition proba- 
bility of a channel to denote the channel itself. Let Cuw be the capacity 
of channel Qxw. 

With the source statistics and channels Qy and Qw given, the 
designer must specify an encoder and a decoder, defined as follows. 

(iv) The encoder with parameters (K, N) is another channel with 
input alphabet S*, output alphabet 2%, and transition proba- 
bility qz(x|s), s©@S8*, x € £%. When the K source variables 
S* = (8, ---, Sx) are the input to the encoder, the output is the 
random vector X¥. Let Y¥ and Z% be the output of channels Q{?” and 

wy, respectively, when the input is KX”. The equivocation of the 
source at the output of the wire-tap channel (corresponding to a 
particular encoder) is 


K N 7 
z WSF |Z"). (2) 
We take A as our criterion of the wire-tapper’s confusion. From the 
system designer’s point of view, it is, of course, desirable to make A 


large. 
(v) The decoder is a mapping 


fo: YN — 8*. (8a) 


Let § = ($1, °° -, Sx) = fo(Y). Corresponding to a given encoder and 
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decoder, the error-rate is 
1x z 
Pe= ed Pr {Si ¥~ Sz}. (8b) 
k=1 


We refer to the above as an encoder-decoder (K, N, A, P,.).* The 

applicability of the above to the system in Fig. 2 should be obvious. 
Next, we say that the pair (R, d) (where R, d > 0) is achievable if, 

for all « > 0, there exists an encoder-decoder (N, K, A, P.) for which 


Oe) > R~ (9a) 
AZzd- gs. (9b) 
PeS.. (9c) 


Our problem is to characterize the set ® of achievable (R, d) pairs. 
Let us remark here that it follows immediately from the definition 
that ® is a closed subset of the first quadrant of the (R, d) plane. 
Before stating our characterization of ®, we digress to discuss a certain 
information-theoretic quantity that plays a crucial role in our solution. 

Consider the channels Qu, Qw, and Quw defined above. Let px(z), 
x € 2X, be a probability mass function and let X be the random 
variable defined by 


Pr {X = x} = px(z), te X. 


Let Y, Z be the outputs of channels Qy and Qyw, respectively, when 
X is the input. For R = 0, let @(R) be the set of px such that 
I(x; Y) 2 R. Of course, @(R) is empty for R > Cy, the capacity of 
channel Qy. Finally, forO < R S Cy, define 


I> 


T(r) £ sup I(X;¥{Z). (10) 


Px CO (R) 


We remark here that, for any distribution px on X, the corresponding 
X, Y, Z forms a Markov chain, so that the definition of mutual infor- 
mation and (4) yield 


I(X;Y|Z) = H(X|Z) — H(XY, Z) 
= H(X|Z) — H(X|Y) = 1(X;Y) —1(X;Z). (1) 


Thus, we can write (10) as 


T(R) = sup J(X;Y|Z)= sup [1(X;Y) —-1(X;Z)]. (12) 
px EO (FR) px EO (R) 


* This should be read as “‘... an encoder-decoder with parameters (K, N, A, P.).”’ 
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As an example, suppose that X = Y = 3 = {0,1}. Let Qa be a 
noiseless (binary) channel, and let Qw be a binary symmetric channel 
(Bsc) with crossover probability po. Then for arbitrary px, 


I(X;Y) — I(X;2Z) = H(X) — [H(Z) — H(2Z|X)] 
= h(po) + H(X) — H(Z) S h(po), 


where h(-) is defined in (3). The inequality follows from the well- 
known fact (see, for example, Ref. 2) that the entropy of the output 
of a Bsc, i.e., H(Z), is not less than the entropy of the input, H(X). 
Further, H(X) = H(Z) if and only if px(0) = px(1) = 3. Since this 
distribution belongs to @(R), for all R,O S$ R S Cy = 1, we conclude 
that, in this case, 

T(R) =h(po), OS RS Cy. (13) 


In Appendix B, we establish the following lemma concerning I'(R). 
Lemma 1: The quantity T(R),0 S R S Cy, satisfies the following: 


(t) The “supremum” in the definition of TL (10) or (12) ] is, in fact, 
a maximum—i.e., for each R, there exists a px € @(R) such 
that I(X; Y|Z) = T(R). 
(2) T'(R) is a concave function of R. 
(it) T'(R) is nonincreasing in R. 
(iv) T'(R) ts continuous in R. 
(v) Cy 2 T(R) 2 Cu — Cuw, where Cy and Cuw are the capact- 
ties of channels Qu and Qu, respectively. 


We can now state our main result, the proof of which is given in the 
remaining sections. 


Theorem 2: The set ®, as defined above, is equal to R, where 
RA (Rd): OSRSCy, OSdS Hs, Rd < Hsl(R)}. (14) 
Remarks: 


(1) A sketch of a typical region @ is given in Fig. 3. In the above ex- 
ample (Qa noiseless and Qw a Bsc), '(R) = h(po), a constant, so that 
the curve Rd = HsI(R) is a hyperbola. Observe that in this case 
the region & is not convex. This is in contrast to the up-to-now essen- 
tially universal situation in multiple-user Shannon theory problems, 
where the solution is nearly always a convex region. Whether or not 
I'(R)/R is always convex, as it appears in Fig. 3, is an open question. 

(2) The points in ® for which R = Cy, correspond to data rates of 
about the capacity of Qi. This is clearly the maximum rate at which 
reliable transmission over Qy is possible. An equivocation at the 
wire-tap of about HsIl(Car)/C is achievable at this rate. An increase 
in equivocation requires a reduction of transmission rate. 
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Fig. 3—Region @. 


(3) The points in ® for which d = Hs are of considerable interest. 
These correspond to an equivocation for the wire-tapper of about 
Hs—.e., perfect secrecy. A transmission rate of 


C,= max R 
(R, Hs)ER 


is therefore achievable in perfect secrecy. We call C; the ‘secrecy 
capacity” of the channel pair (Qu, Qw). The following theorem 
clarifies this remark. 


Theorem 3: If Cu > Cw, there exists a unique solution C'; of 
C, = T(C,). (15) 
Further, C satisfies 
0< Cy — Cuw ST(Cmu) SC; S Cu, (16) 
and C, is the maximum R such that (R, Hs) € @. 
Proof: Define G(R) = T(R) — Rk, OS R S Cy. From Lemma 1 (0), 
G(Cu) = T(Cu) — Cu S 0, 
and 
G(0) = T(0) = Cu — Cuw > 0. 


Since by Lemma 1, (777) and (zv), G(R) is continuous and strictly 
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decreasing in R, a unique C, € (0,Cy] exists such that G(C;) 
= T'(C,) — C, = 0. This is the unique solution to (15). Inequality 
(16) follows from C, € (0, Cy] and Lemma 1, (227) and (v). Finally, 
from (15) and (16) we have (C;, Hs) € ® = &. Also, if (Ri, Hs) € &, 
then HsR, S HsI(R:) so that G(R) = 0. Since G(R) is strictly 
decreasing in R, we conclude that Ri < C;. Thus, C; is the maximum 
of those R for which (Ri, Hs) © ®, completing the proof. 

(4) It is clear that the source statistics enter into the solution only 
via the source entropy Hs. We also remind the reader that the fairly 
simple extension of Theorems 2 and 3 to a stationary, ergodic source 
is given in Appendix C. 

(5) If we define P.w, the ‘‘wire-tapper’s” error probability, as the 
error rate at a decoder built by the wire-tapper [defined analogously 
to (8) ], then it follows from Fano’s inequality (see Appendix A) that 


A S h(Pew) + Pew log |8|. 


Thus, a large value of the equivocation A implies a large value of 
P.» (which the system designer will find desirable). 


Ill. PROOF OF THEOREM 2 FOR A SPECIAL CASE 


In this section we prove Theorem 2 for the very special case dis- 
cussed in Section I. All alphabets 8, X, Y, 3 are equal to {0, 1}. The 
source {S,} satisfies Pr {S, = 0} = Pr {S, = 1} = 4. Channel Qy, is 
noiseless, i.e., Qar(y|x) = 6;,y; and channel Qy is a Bsc with crossover 
probability po (0 S po S 3), 1€., 


Qw(z|y) = (1 — po)dy,2 + po(l — 4,,2). (17) 
We show here that (R, d) is achievable if and only if 


Since, for this case, '(R) = h(po), this result is a special case of the 
as-yet-unproven Theorem 2. We begin with the converse (‘‘only if’’) 
part of the result. Let S*, KX”, Z* correspond to an encoder-decoder 
(NV, K, A, P.) (note that YY = X”). Then, making repeated use of 
the identity H(U, V) = H(U) + H(V|U), we can write (dropping 
the superscript on vectors) 

KA = H(S*|Z*) = H(S, Z) — H(Z) 
H(S, X, Z) — H(X|S, Z) — H(Z) 
= H(Z|X, 8) + H(X, 8) — H(&|S, Z) — H(Z) 


= H(Z|X) + H(S|X) + H(X) — H(&|S, Z) — H(2) 
b) 
= Nh(po) + H(S|X) + CH(X) — H(@Z)] — HIS, Z). (19) 


These steps are justified as follows. 
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(a) From the fact that (S, X, Z) is a Markov chain and (4), so that 
H(Z|X, S) = H(Z|X). 

(b) Since X, Z are the input and output, respectively, of a Bsc, 
H(Z|X) = Nh(po), regardless of the distribution for X. 


Now from Fano’s inequality [use ineq. (78) with V = X], we have 
H(S|X) Ss Kh(P.). Further, the entropy of the output of a Bsc = the 
entropy of the input [this follows from Mrs. Gerber’s lemma (Ref. 2, 
Theorem 1)], so that H(X) — H(Z) <0. Finally, H(X|S, Z) 2 0. 
Thus, (19) yields for any encoder-decoder (K, N, A, P.), 


KA S Nh(po) + KA(P.), 
or 


K [A — (PI S Alpe. (20) 


Now suppose that (R, d) is achievable. It follows from the ordinary 
converse to the coding theorem (Ref. 1, Th. 4.3.4, p. 81) that 
R <= Cy = 1. Further, since A S Hs = 1, we conclude that d < 1. 
Finally, if we apply (20) to an encoder-decoder (N, K, A, P.) that 
satisfies (9) with e > 0 arbitrary, we have 


(R — e)[(d — «) — A(e)] S h(po). 


Letting «—0 yields Rd S h(po). Thus, we have established the 
converse of Theorem 2, i.e., that an achievable (R, d) must satisfy (18). 

We begin the proof of the direct half of Theorem 2 with a digression 
about group codes for the ssc. Let G € {0, 1}* be a group code (i.e., 
a parity check code) as defined for example in Ref. 1, Chapter 6, or 
Ref. 3, Chapter 4. The group code G has M = 2%/|G| cosets. Denote 
the cosets by Co = G, Ci, Co, +++, Cu_i. Of course, the cosets are 
disjoint and 


Ge 10.10%. 


1=0 


Let \ be the word error probability when group code G (or for any of 
the cosets) is used on a BSC with crossover probability po, with maxi- 
mum-likelihood (minimum distance) decoding. Thus, for each coset 
C;,0 <7 S M —1, there exists a decoder mapping D;: {0, 1}” > C,, 
such that if X* is the input to a Bsc with crossover probability po, and 
Z*® is the corresponding output, then for allx C Ci,0 S75 M —1, 


Pr {D;(Z") 4 X"|X¥ = x} = 2. 
Thus, regardless of the probability distribution for X”, 
Pr {D,(Z¥) ¥ X"|X" E C;} = d. 
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Letting y(x) = 7, forx © Ci, 0 Si S M — 1, we have, from Fano’s 
inequality [use ineq. (76) with U = X¥, V = Z., U = D,(Z%)],, 


H(X*|Z", p = 7) S h(A) + Alog [Ci]. 
Therefore, for any X distribution (which induces a distribution of w), 
H(XN|Z", py) Sh(A) + Alog [G|. (21) 


We conclude this digression by stating as a lemma the well-known 
result of Elias that there exists a group code for transmitting reliably 
over a BSC at any rate up to capacity. A proof of this result can be 
found in Ref. 1, Section 6.2. 


Lemma 4: Let «. > 0, r < 1 — h(po) be arbitrary. Then, provided N is 
sufficiently large, there exists a group code G of block length N with 
\G| = 2%", such that, on the Bsc with crossover probability po, the error 
probability \ S 1. 

We now prove the direct half of Theorem 2 for our special case by 
showing that any (R, d), where R is rational, which satisfies 


R-d = h(po); (22a) 
OSd<l, (22b) 
OSRS1 (22c) 


is achievable. Thus, for (R, d) satisfying (22), and arbitrary « > 0, 
we must show the existence of an encoder-decoder (N, K, A, P.) that 
satisfies (9). We now proceed to this task. 
Let K, N satisfy 
K 
v= R. (23) 
Let G be a binary group code with block length N and with |G| 
= 2\™-*), Thus, G has M = 2* cosets {C;}?£o. We can assume that 
the set S* = {0,1}* is the set of integers {0,1,---, MZ—1}. We 
construct the encoder such that when the source vector S* = 7,* the 
encoder output X” is a randomly chosen member of coset C;—i.e., 
pa yee IN on 9-\V-K) forx EC; 
Pr {XY =x|S=7} = 4(C.| |G ; 7 
0, ce C,, 


0 <i M —1.Since S¥ is uniformly distributed on {0, 1, ---, M@ — 1}, 
X* is uniformly distributed on xX” = {0, 1}. Thus, in particular, 


H(X") = H(Z*) = N, (24) 


“This is an abuse of notation. A more precise statement is that S* is a binary 
representation of 7. 
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where, as always, Z” is the output of the wire-tap channel when X” 
is the input. Also let us observe here that the quantity y(X*), defined 
in the above digression, is identical to S*. Thus, (21) yields 


H(K*|Z*, S®) S A() + (N — KB), (25) 


where X is the error probability for the group code G. 
We now turn to the decoder. Letting D(y) = 7, when y € Ci, we 
conclude (since the channel Q,, is noiseless) that 


P, = 0. (26) 


Since (23) and (26) imply (9a) and (9c), it remains to show that a G 
exists such that the resulting encoder-decoder will satisfy (9b). 

We now invoke (19), which is valid for any encoder-decoder. 
Substituting (24) and (25) into (19), and invoking (26), which implies 
H(S|X) = 0, we obtain 


A> =(z ) io = ne (Re - 1): (27) 


Now, from (22a) and (23), we have 


N a h(po) = 
K h(po) a R a d, 





and from (28), 


Thus, (27) yields 
ASe= | Ae +a(9-3)]- (28) 


Finally, since from (23) and (22a) we have 

|G] = QN-K < QND-h(o)/a), 
we can invoke Lemma 4 with r = 1 — h(po)/d <1 —h(po) [from 
(22b) ] to assert the existence of a group code G with ) sufficiently 


small to make the term in brackets in (28) Se. Then A2d-— ¢, 
which is (9b). This completes the proof of the direct half. 


IV. CONVERSE THEOREM 


In this section, we establish the converse theorem that the family 
of achievable rates ® is contained in ® as defined in (14). Suppose that 


WIRE-TAP CHANNEL ~ 1367 


(R,d) © &. That R S Cy follows from the ordinary converse to the 
coding theorem (Ref. i Theorem 4.3.4, p. 81). That d S Hs follows 
from 


= i KIFN) < 1 K\) — 

= RAS [ZX) < ZS ):= Ae: 
Thus, it remains to show that Rd S HsI(R). We do this via a lemma, 
the proof of which is given at the conclusion of this section. 
Lemma &: Let S*®, X%, Y%, Z% correspond to an encoder-decoder 


(N, K, A, P.). Then 


, «K ie S 
(1) ylA—d(P)IS v& T(Xnj Yn|Zn, ¥"™), (29a) 


Gi) F[Hs— 8 PI] SH LUI), 9b) 


where 


6(P.) = h(P.) + Pe log |S}, (29¢) 


and where the n = 1 term in the summations of (29a, b) ts given the 
obvious interpretation—t.e., that I(X1; Y1|Z1, Y°) = [(X1; ¥1|Z1), ete. 


Now for n = 2, 38, ---, N, any y © Yy"}, set 
an(y) = (Xn; Yn/Y"7" = y). (30a) 
Also let 
a= I(X1; Y,). (30b) 


It follows from the definition of ® (2) in Section II that the distribution 
pi, defined by 


p(t) = Pr{Xi=2}, re a, 
belongs to @(a1). Similarly, for2 S$ n S N, with y € Y*" fixed, define 
pay(t) £Pr{X,=2/Ye"=y}, cea. 


Then pny € PLen(y) ]. Thus, from (10) and the fact that channels 
QY and Q are memoryless, 


Tax) 2 1(X1; Yi|Z,), (31a) 
andfor2snSN,y€ yr, 
TLan(y)] 2 (Xn; Yul Zn, Y" = y). (31b) 


It follows that the right member of (29a) is (giving the n = 1 term 
the obvious interpretation) 
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(32) 


=/ 
=] 
rg 

ey 
re 

3 
L 

I 
= 
a 

g 

3 

ae 

co 
es 
LJ 


POG «| yr) 


Step (a) follows from (81), step (b) from the concavity of ! [Lemma 
1(7z) ], step (c) from the definition of an, and step (d) from (29b) and 
the monotonicity of T [Lemma 1 (777) ]. Applying (29a) to (32) yields 


Corollary 6: For any encoder-decoder (N, K, A, P.), 


~ EA (Pj) ST | us - P|: (33) 


We now show that, if (R, d)€ @, then Rd S Hsl(R). Let 
(R,d) © ®, and let ¢ > 0 be arbitrary. Apply Corollary 6 to the 
encoder-decoder (N, K, A, P.) that satisfies (9). Inequalities (83) and 
(9) yield 

(R — e)[(d — e) — 6(c)] S HsI[(R — «) — &(e)]. (34) 
Letting « — 0 and invoking the continuity of ! [Lemma 1 (iv) ] yield 
Rd S Hs (R), completing the proof of the converse. It remains to 
prove Lemma 5. 
Proof of Lemma 5: 
(7) Let S*, X*, Y%, Z* correspond to an encoder-decoder (N, K, A, P.). 
First observe that 
n 


eH (S*|2%, ¥") < LH (S*|Y") 


< h(P,) + P, log (|8| — 1) = 8(P.). (35) 
Inequality (a) follows from Fano’s inequality [use (78) with V = Y” ]. 
Next, using the definition of A (7) and (85), write 
KA = H(S*|Z%) s H(S*|Z%) — H(S*|Z", Y¥) + Ké(P.) 
I(S¥; Y" |Z") + Ké(P.) 
I(X*;Y¥"|Z") + K6(P.). (36) 


WA Il 
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The last inequality in (36) follows from the data-processing theorem, 
since given ZN = z, (Y%, X¥, S*) is a Markov chain (Appendix A). 
Transposing the K6(P.) term in (36) and continuing: 
K[A — 4(P.)] < 1(K¥; ¥¥|Z¥) 
= H(X"|Z") — H(X*|Z¥, Y%) 
(a) 
= H(X"|Z%) — H(X*|Y%) 
= 1(K"; ¥¥) — 1(K"; 2) 
H(Y") — H(ZN) + H(Z*|X*) — H(Y" |X) 
N 
DX CH(Y.|¥"") — H(Z,|Z") 
n=1 


is 
— 


¥ LA(Pa|¥) — HZa|Z, YO) 


IA& 


+ H(Z,|X1) — H(V2|Xn)] 
 CH(¥a|¥) — A(Za/¥) + AZ| Xn, Y) 


n=1 


be 
le 


+ H(¥n|Xn, Y™)] 
- = LEX Kalk) = 1a Zul) 


2 2 [H(X»|Zn, ¥*) — H(Xn| Ya, YY] 
7 > [H(Xa|Zn ¥°) — H(Xal Yn, Zn YY] 
ss > 1X, -¥,|Zn Y"-). (37) 


The steps in (37) that require explanation are: 


(a) that follows from the fact that KX”, YY, Z¥ is a Markov chain 
and (4); 
(b) that follows from the standard identity 


HU") = & HU,|U-9, 


and the fact that channels QY and QM, are memoryless; 
(c) that follows from the fact that conditioning decreases entropy; 
(d) that follows on applying (4) to the Markov chains (Z*—!, Y"~|, 
Zn), Oda Xn, Y Zn); 
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(e) that follows from the fact that, given Y"—!, (Xn, Yn, Zn) is a 
Markov chain. 


Since (37) is (29a), we have established part (2) of Lemma 5. 
(at) With S*, X”, Y”, Z*, as in part (2) write 
H(S*) = 1(8*}¥") 1 (S*|Y*) 
LRP EY") OP), (38) 


IA Il 


where the inequality follows from the data-processing theorem (since 
S*, XV, Y", is a Markov chain) and from Fano’s inequality as in (35). 
Since H(S*) = Kds, (38) yields 


KBs — 6(P.)] S 1" YX") 
= s CH(Y.|Y") — H(¥.|X»)] 
e y (H(Y,|¥">) — H(Y¥,|Xn, ¥*-)] 
Z pee Y.|Y™). (39) 


Step (a) follows on application of H(Y”) = >°, H(Y,|Y"~'), and the 
memorylessness of channel Q{”, and step (b) from the fact that 
YY", X,, Yn is a Markov chain. Inequality (89) is (29b), so that the 
proof of Lemma 5 is complete. 


V. DIRECT HALF OF THEOREM 2 


In this section we establish the direct (existence) part of Theorem 2, 
that is, R C BR. The first step is to establish two lemmas that are 
valid for any encoder-decoder as defined in Section IT. 


Lemma 7: Let S¥, XX, YY, Z correspond to an arbitrary encoder-decoder 
(N, K, A, P.). Then 


Ka & H(S*|Z%) = H(S*) + I(K%; Z*|S*) — (KX; Z%). (40) 
Proof: By repeatedly using the identity H(U, V) = H(U) + H(V|U), 
we obtain (we have omitted superscripts) 

KA = H(8|Z) = H(S, Z) — H(Z) 

= H(S, Z,X) — H(&|S, Z) — H(Z) 

= H(Z|X,S) + H(X%,S) — H(&|S, Z) — H(Z) 

= H(Z|X, S) + A(S) + [H(4|S) — A(K|S, Z)] — A(Z) 

= H(S) + I(%;Z|S) — LH(Z) — H(Z|X, S)]. (41) 
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Now, since S, X, Z is a Markov chain, H(Z|X,S) = H(Z|X) [by 
(4) ]. Thus, the term in brackets in the right member of (41) is J(X; Z), 
completing the proof. 

We now give some preliminaries for the second of the two lemmas. 
For the remainder of this section we take the finite set X to be 
{1,2,---, A}. Let X* be a random variable that takes values in X 
with probability distribution 


Pr {X* = 1} = px(i), 1SISA. 


Let Y* and Z* be the output of channels Qu, and Qyyw, respectively, 
when X* is the input. As always, Qurw is the cascade of Qy and Qy, 
so that X*, Y*, Z* is a Markov chain. Next, for 1 $7 A, and 
x € X% define 


# (1, x) S card {n:2n = 2} 
= number of occurrences of the symbol 7 in the 
N-vector x. (42) 





For N = 1, 2, ---, define the set of ‘‘typical’’ X sequences as the set 
T* = T*(N) = xe 9CN EOD _ p20) <ty,1SiS A}, 
(43a) 
where 
by = N-+. (43b) 


Let us remark in passing that the random N-vector X** consisting of 
N independent copies of X* satisfies E# (7, X*”) = Npx(2), and 
Var [# (7, X*¥)] = Npx()[1 — px()], for 1 SiS A. Thus, by 
Chebyshev’s inequality 


A 
Prix TN) Ss Xu Pr {|# (2, X*) — Npx(2)| > Néw} 
a 1 
< > Var [# (7, X*) ]/N?63, = (= )=0, 44 
& Var [# , XY a (44) 
as N >, 
We can now state the second of our lemmas. We give the proof at 


the conclusion of this section. 


Lemma 8: Let KX, Z* correspond to an arbitrary encoder and let X*, Z*, 
T* correspond to an arbitrary px as above. Then 


py E&*; 2") S I(X*, 2) + (log 4) Pr (K" E T#(N)} + AlN), 
where fi(N) —0, as N > &. 
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Lemma 8 implies that, if the encoder is such that with high proba- 
bility XY € T*, then (1/N)I(X*;Z%) cannot be much more than 
HO. Gare Ag 

Lemmas 7 and 8 hold for any encoder-decoder. Our next step is to 
describe a certain ad-hoc encoder-decoder and deduce several of its 
properties. We then show that when the parameters of the ad-hoc 
scheme are properly chosen, the direct half of Theorem 2 will follow 
easily. 

We begin the discussion of the ad-hoc scheme by reviewing some 
facts about source coding. With the source given as in Section II, 
for K = 1, 2, ---, there exists a (‘‘source encoder’’) mapping Fz: 
Sk — {1, 2, ---, M}, where 

M = 2K#s(tox). (45) 


and 6x = K7-}. Let Fp: {1, 2, ---, M}—S8* be a (“source decoder’’) 
mapping, and let 


P&® = Pr {FpeF's(S*) ¥ S¥} 


be the resulting error probability. It is very well known that there 
exists (for each K) a pair (fz, Fp) such that, as K >, 


P§& = Pr {Fp(W) # S*} — 0, (46a) 
where 
W = Fx (S*). (46b) 
We will design our system to transmit W using an (Fz, Fp) that 
satisfies (46). 
We now turn to our ad-hoc system. (Refer to Fig. 4.) The source 


output is the vector S*, and the output of the source decoder is 
W = Fr(S*). Let 


q: 2 Pr {W = Fx(S*) =i}, 1Si< M. (47) 











A A 
W=F,(S) source | SK = Fo(W) 


DECODER 





CHANNEL| YN 
SOURCE CHANNEL 
SOURCE ENCODER ENCODER 





ZN 


Fig. 4—Ad-hoc encoder-decoder. 


WIRE-TAP CHANNEL = 1373 


Next, let M@, = M.M be a multiple of VM to be specified later. Let 
{Xm} i" 


be a subset of 2%. Clearly, {x,,} can be viewed as a channel code for 
channel Q® or channel Q{&%),. The channel encoder and decoder in 
Fig. 4 work as follows. The channel encoder and decoder each contains 
a partition of {xn}%" into M subcodes Ci, Co, ---, Cur, each with 
cardinality M,. Assume that 


C; = {X(-1) Mo41)° hiner) Xia}, 1sxi<xM. (48) 


When the random variable W = 7, then the channel encoder output 
X* is a (uniformly) randomly chosen member of the subcode C;. Thus, 
forlsisM,1sj8s ™, 


Pr {X¥ = X¢-yatyas|W = 7} = UM’ (49a) 


and 





Pr {X¥ = xXG_narnis} = (49b) 


qi. 
Ms. 
Now the set {x,}/" can be thought of as a channel code for channel 
™ with prior probability distribution on the code words given by 
(49b). A decoder for the code is a mapping G: Y% — {x,,}?#" and the 
(word) error probability is 


\ = Pr {G(y¥) 4X}, (50) 


where Y” is the output of Q{”, when the input X” has distribution 
given by (49b). We assume that the channel decoder in Fig. 4 has 
stored the mapping G. When the channel output is y € Y%, the channel 
decoder computes G(y). When G(y) € C;, the channel decoder output 
is i, 1 <i M. Letting W be the output of the channel decoder, 
we have 

Pr {W~W} <) 


The final step in the system of Fig. 4 : the emission by the source 


decoder of S¥ = Fp(W), where Fo: {1, -, M} — SX is chosen so 
that (46) holds. We have 
Pr {S = 8S} = Pr {S = Fp(W)} 


IV 


Pr {S 


Fp(W); W = W}. 
Thus, 
P. S Pr{S #8} S$ Pr{S # Fp(W)} 
+Pr{W4#W}) <5 P+ x. (51) 
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Next, let us observe that each of the subcodes C; can be considered 
a code for channel Q{%}, with M, code words and uniform prior distri- 
bution on the code words. Let \; be the resulting (word) error proba- 
bility for code C; (1 S$ 7 S M) with an optimal decoder, and let 


i) 


f 


qidi. (52) 


~! 
\| 


I 
pu 


We now establish 
Lemma 9: For the ad-hoc encoder-decoder defined above 
I(&*; ZY|S*) = log Mz — [h(A) + d log Me]. 


Proof: Let S* be such that W = Fe(S*) = 7. Then the channel 
input X*” given W = 7 has distribution given by (49a), ie., X” is a 
randomly chosen member of C;. Since \; is the error probability for 
code C;; used on channel Q{},, Fano’s inequality [use (76) with U = X¥, 
V = Z’, U = the decoded version of ZY when code C; is used] yields 


H(X®|Z", W = i) S AAD) + As log M;, 
and, since H(X"|W = 7) = log M2, we have 
I(&"; ZY|W = 7) 2 log Mz — h(Ai) — i log M2. 


Averaging over 7 using the weighting {q:}, and using the concavity 
of h(-), we have 


I(&";ZN|W) = log M, — [h(A) + A log Me]. (53) 
Finally, since S, W, X, Z is a Markov chain, (4) yields 


I(X;Z"|W) = H(Z|W) — H(Z|XW) 
= H(Z|W,S) — H(Z|X) 
= H(Z|W,S) — H(Z|X,$) 
< H(Z|S) — H(Z|X, 8) = I(K";Z*|S). (54) 


Inequalities (53) and (54) imply Lemma 9. 
We are now ready to combine the above lemmas as: 


Corollary 10: Let px be an arbitrary probability distribution on X, and 
let Tx(N), X*, Y*, Z* be as defined above (corresponding to px). Assume 
that S*, X”, Y", Z" correspond to the above ad-hoc encoder-decoder with 
parameters N, K, M, My, Mo, x, ». Let P. and A correspond to this 
ad-hoc scheme. Then 

| ee ores a} (55a) 
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h(X) _ A log Mz 
N  N- 

— (log A) Pr {XY € TX(N)} — fi(N), (55b) 
where fi(N) ~ 0 as N->~. 


Proof: Inequality (55a) is the same as (51). Inequality (55b) is ob- 
tained by substituting the results of Lemmas 8 and 9 into (40) and 
using H(S*) = KH. 

Finally, we are ready to prove the direct half of Theorem 2. We do 
this by showing that any pair (R, d), which satisfies 


K 1 _ , n=. 
vis + 7 log Me LX, 2") 


=I 
> 
IV 


R-d = Hs (R), (56a) 
OS RSCy, (56b) 
Os d Ss Hs, (56c) 


is achievable. Thus, for (R, d) satisfying (56) and for arbitrary « > 0, 
we show that our ad-hoc scheme with appropriately chosen parameters 
satisfies (9). To begin with, choose K, N to satisfy 


ALF. 
N Hg 
(Assume that R/Hs is rational.) Note that (57) implies (9a). Also, let 


px be a distribution on X that belongs to @(R) and achieves '(R)— 
that is, 


(57) 


LOT. yy Sh, 
LXE Yt) LZ) Sd | 2) = Te), (58) 
where X*, Y*, Z* correspond to px. We now assume that an encoder- 


decoder is constructed according to the above ad-hoc scheme with 
the parameter* 





Miu Bao: [1 kee y*) — | (59) 


where X*, Y* correspond to the above choice of px. With this choice 
of M,, and with M given by (45), we have 


oie, x. ps) Ky, _* _ et |, 
Note that, from (57), 


* Assume that the right member of (59) is an integer. If not, a trivial modification 
of the sequel is necessary. 
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i = x. px, _ _K _ 
y 108 M2 = L(X ~Y*) vis yy fi sox oH s 
(a) eR 
_— *. Ve) RP _ ae 
TOE) = Ba Riga 
(Rd/Hs) eh 
SAX SS Rie 
se") — “G/Hs) < 2H 
(b) eh 
Ss 
= 1(X*; ¥*) — 1(X*; Y*|Z4) — Rog — 
Ss 
(c) eh 
a *. TR) = : 
U(X"; 2") = Riz’ — 55 (61) 


Step (a) follows from (57), step (b) from (56a) and (56c), and step (c) 
from the fact that X*, Y*, Z* is a Markov chain—see (11). 

Let us now apply Corollary 10 to the ad-hoe scheme with the above 
choice of M,, M>, and with the above choice of px. Inequality (55a) 
remains 


P.S PH) +), (62) 
and substituting (60) into (55b) yields 


(RA)/Hs 2 I(X*; Y*) — I[(X*; Z*) — fr(N) 
= [(R) — f2(), (63a) 
where 
fa(N) = nO) 4 dlog M: M, 





ae 
+ (log A) Pr {KY € T*(N)} + fi(V).  (68b) 


Now observe f2(V) and X depend on the choice of the set {x,}?" 
The following lemma asserts the existence of a {z,,} such that these 
quantities are small. Its proof is given at the end of this section. 


Lemma 11: With px and M,, M, as given above, there exists for arbitrary 
N a set 
{Xm }me 
such that 
Pr {X” € T*(N) 


>I ca — 


i < f.(N), (64) 
where f3(N) 0, as No. 
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Now let the set {x,,}#" in the ad-hoc scheme be chosen to satisfy 
(64). Then, from (62) and (64) [using the fact that P&*— 0, as 
K-> (46) ], we can choose N (and K = NR/Hs) sufficiently large 
so that 

P= 6 


this is (9c). It remains to establish (9b). But from (64) with N suffi- 
ciently large, we can make 


— X Mog M M, eR 


+ (log A) Pr {X*” € T*(N)} + fi(N) S 





Then (63) and (56a) yield 
HsT(R) 
Sh ee Ns = — 
— R d €, 


which is (9b). Thus, (R, d) is achievable and the proof of the direct half 
of Theorem 2, ie., ® C R, is complete. It remains to prove Lemmas 
11 and 8. 


Proof of Lemma 11: We begin with some notation. For x € 9, let 


col ETI), 
He) i otherwise. 2) 
Also for a given set {xn}1", let X (x1, +--+, Xar,) be the error proba- 


bility that results when { Xn} is used as a channel code for channel 
Q® with prior probabilities (49b) when code word x,, is transmitted 
and when maximum liklihood decoding is used. Thus, 


M tM. Qi 
A = a > =— \™ (xy, -o+, Xu). 
t=1 m=(i-1)M2+1 2 


Further, with \; defined as above as the error probability for code 
C; on QU, write iG = Aarw (X G1) My41) aa Xim,) _ Amuw (Ci), so that 
the dependence of i; on C; is explicit. We have 


M 
A= zy gai = DL qiduw (Ci). 
Finally, define 
{XX € T(N)} +AHA 


t~M2 # 
. , LH Xm) + A™ (X41, ae Xu.) | 


®(X1, °° Xu) 2 — 





Pr 
M 
— 


m=(t—1)Me 41M 


++ > qgiAmw(C:). (66) 


Now suppose that the set {z,,}{ is chosen at random, with each x,, 
chosen independently from 9%, with probability distribution p¥ (x) 
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= [[*1 px(z,). We establish the lemma by showing that E@ < F3(N). 
Now observe that, from (59), (1/N) log M1 is bounded below I(X*,Y*). 
Also from (61), (1/N) log Mz is bound below I(X*; Z*). It follows 
from the standard random channel-coding theorem (see, for example, 
Ref. 1, Theorem 5.6.2) that EA™, EX\uw S fa(N) 0, as No. 
Further, Eu = Pr {X* € TX(N)} S fs(N) 0, by (44). Thus, H® 
< 2f,(N) + fs(N) & fs(N) > 0. Hence the lemma. 


Proof of Lemma 8: Here too we begin with some notation. Let p be a 
probability distribution on 9, and let J(p) be the mutual information 
between the input and output of channel Qirw when the input has 
distribution p. It is known (Ref. 1, Theorem 4.4.2) that J(p) is a 
concave function of p. Let u(x) be as in (65), and write (for any 
encoder-decoder) 


1 N-7IN 1, N N)\. FN 
5 (RN; 2") = 5 TOR", w(R¥) ; 2] 
— 2™|u(X*)] + 5 Ta (K") 5 29] 
= Pr wR) = TR; Zu) = A 
+ 5 Tlu(&); 2"). (67) 
Now 


© Pr (u(&®) = LIEK"; 2 |w(X¥) = 1] 
< (log A) Pr {X¥ € T*(N)}, (68) 
and 


Hy Mu(K") 2] 5 He) so (69) 


One term remains in (67). Using the memoryless property of channel 
Oy (Ref. 1, Theorem 4.2.1), we have 


= (8%; 2"|y = 0) $5 Dy I(XnjZnlu = 0) 


(pa) $4 ( ap 2, pa), (70a) 


where p, is the probability distribution for X, given uw = 0, Le 
forl SiS A, 


a(t) = YX 8254 Pr (X¥ = x[K¥ € T*}. (70b) 
xCT* 


I 
=| 
ive i 0 


The last inequality in (70a) follows from the concavity of J. From 
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(70b), 
& 


pW) AHL mo) = Pri =xixe ry 22. cn) 


n=l 
The definition of T* (43) and eq. (71) yields 
|p(t) — px(t)| S dv 0, asN—>o. 
Since J(p) is a continuous function of p, we have 
|9(@) — S(px)| Sg) 70, asN om. (72) 
Substituting (72) into (70a), we obtain 


a Pr (w= O}L(K"; 2¥|u = 0) S (os) + g(N) 
= 1(X*; 24) + g(N). (78) 


Finally, setting fi(V) = (1/N) + g(N), and substituting (68), (69), 
and (73) into (67) we have Lemma 8. 
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APPENDIX A 
The Data-Processing Theorem and Fano’s Inequality 
Let U, V, U be discrete random variables that form a Markov 

chain. Then the data-processing theorem can be stated as 

H(U|V) S$ H(U|D), (74a) 
or equivalently 

I(U; V) 2 I(U; U). (74b) 
Inequality (74a) follows on writing 


(a) a. (b) ‘A 
H(U|V) = H(U|V,U) s H(U|0), 
where step (a) follows from (4), and (b) from the fact that conditioning 
decreases entropy [ Ref. 1, eq. (2.3.13) ]. 


‘ 
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Next, let U, V, U be a Markov chain as above, but now assume 
that U, U take values in u(|U] S ). Let 
h=Pr{U # U}. (75) 
Fano’s inequality is 
H(U|V) S hQ) + Alog (|U] — 1) SHO) + Alog Ul. (76) 
To verify (76), define the random variable 
wes 0y= fh U5 8 


and then write 


_~ 
~ 
~ 


H(U|U) < H(U,®|0) 

H(®|U) + H(U|U, ®) 

H(®) + H(U|U, ®) 

H(&) + Pr {@ = 0}H(U|U, ® = 0) 
+ Pr {@ = 1}H(U|U, @ = 1) 


A(A) + (1 — A)-0 + AN(U| UV, ® = 1) 
S A(d) + Alog ([4] — 1) SAO) + Alog [ul, 


which is (76). Step (a) is (74a), and step (b) follows from the fact 
that, given @ = 0, then U = U, so that H(U|U, ® = 0) = 0, and 
step (c) from the fact that, given 6 = 1, U takes one of the |u| — 1 
values in U excluding U. : 

A variation of Fano’s inequality is the following. Let S*, V, S* 
be a Markov chain where the coordinates of S¥ and §* take the 
values in the set 8. Let 


H(U|V) 


HA 


ll IIA 


és 
Ie 


Pa = Pr {Sx ¥ S;} (77a) 

and 
1 $ 77b 
Pe = K a Pex. ( ) 


We will show that Fano’s inequality implies 
EH(S*|V) SAP.) + Pelog (|S|-1) 28(P). (78) 


To verify (78), write 


lA 


1 K 
ZHS*IV) 


IAS 
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which is (78). Step (a) is a standard inequality, step (b) follows on 
applying (76) to the Markov chain S;:, V, S:, and step (c) from the 
concavity of 6(-). 


APPENDIX B 
Proof of Lemma 1 

(t) With no loss of generality, let x = {1,2,---,A}. Any 
probability distribution px can be thought of as an A-vector 
Pp = (pi, p2, --+, pa). Since 1(X; Y) is a continuous function of px, 
the set ®(R) is a compact subset of Euclidean A-space. Since J(X ; Y |Z) 
is also a continuous function of px, we conclude that [(X; Y|Z) has 
a maximum on @(R). This is part (2). 

(22) Let 0 S Ri, Ro. S Cy, and 0 S$ 6S 1. We must show that 


T[oR: + (1 — 6)Rz] = 6 (Ri) + (1 — #)T(R2). (79) 


For « = 1, 2, let p; € ®(R:) achieve ['(A,). In other words, letting 
X;, Y;, Z; correspond to p;, 2 = 1, 2, then 


(Xi, Y¥) 2 R: I(X:, Y:|Z:) = T(R,). (80) 


Now let the random variable X be defined as in Fig. 5. Forz = 1, 2, the 
box labeled “‘p;”’ generates the random variable X; that has probability 
distribution ‘‘p;.”’ The switch takes upper position (“position 17’) 
with probability @ and the lower position (“position 2’’) with proba- 
bility 1 — 6. Let V denote the switch position. In the figure, V = 1. 
Assume that V, Xi, X2 are independent. As indicated in the figure, 
X = X;, when V = 7,7 = 1, 2. Now 


1(X:Y) = H(Y) — H(¥|X) = A(Y) — H(Y|X, V) 
2 A(Y|V) — A(Y |X, V) = 1(X; Y|V) 

= 61(X;Y|V =1)+ (1—98)1(X;Y|V = 2) 

6I(X1; Yi) + (1 — 0)1(X2; Yo) 


OR: + (1 — @)Re. (81) 


= 
IVE 


Fig. 5—Defining the random variable X. 
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Step (a) follows from the fact that V, X, Y is a Markov chain and 
(4). Step (b) follows from (80). Inequality (81) implies that the 
distribution defining X belongs to @[6R; + (1 — 6)R,]. Thus, from 
the definition of I, 


Toki + (1 — 6)R.] = T(X; YZ). (82) 
Continuing (82) and paralleling (81), we have 


Tek, + (1 — @—)R.] = H(VY |Z) — H(Y|XZ) 

= H(Y|Z) — H(Y|XZV) 

> H(Y|ZV) — H(Y|XZV) 
I(X;Y|ZV) = 0l(X;¥|Z, V = 1) 
+ (1 — @I(xX;Y|Z, V = 2) 
6l(X1; Yi{Z1) + (1 — 0)1(X2; Y2|Zz) 
= 6 (Ri) + (1 — aT (R,), 


which is (79). This is part (72). 

(zz) This part follows immediately from the definition of T'(2) 
(10), since @(R) is a nonincreasing set. 

(7v) Since I'(R) is concave on [0, Cy, ], and nonincreasing, it must 
be continuous for 0 S$ R < Cy. Thus, we need only verify the con- 
tinuity of ['(R) at R = Cy. Let p be a probability distribution on X 
viewed as a vector in Euclidean A-space, as in the proof of part (7). 
Let J(p) and S(p) be the values of J(X; Y) and I(X; Y|Z), respec- 
tively, which correspond to p. 9(p) and 5(p) are continuous functions 
of p. 

Now let {R;} be a monotone increasing sequence such that 
R; — Cy, and R; S Cy. We must show that, asj > ~, 


T'(R;) > (Cy). (83) 
Now from the monotonicity of T'(R), lim;.., ['(2;) exists and 


lim P(R,) = T(Cu). (84) 


It remains to verify the reverse of ineq. (84). Let {p;}7? satisfy 
s(p;)) 2 Rk; (pi) = T(R,), (85) 


for 1 <j < «. Since the set of probability A-vectors is compact, 
there exists a probability distribution p* on X such that for some 
subsequence {p;,}f=1 


is _ * 
lim p;, = p*. 
ko 
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It follows from the continuity of 9(-), and (85) that J (p*) = Cm, so 
that p* € @(Cy). Therefore, from the continuity of g(-), and (85), 
we have 


A A (a) 
lim T(R;) = lim T'(R;,) = lim g(p;,) = J(p*) ST(Cu), (86) 
pro kaw k->0 


where step (a) follows from p*C@(Cy,). Inequalities (84) and (86) 
yield (83) and part (zy). 
(v) From (12), 
T(R) = sup (1(X;¥) —1(X;2Z)] 
px EO (R) 
= sup I(X;Y) S Cu, 
px EP (R) 
which is the first inequality in part (v). Also, using (12), 
T(Cu) = sup (I(X;Y) — 1(X;2)] 
Px EP (Cur) 
sup [I(X; Y) — Cuw | = Cue Cuw. (87) 
Px GO (C1) 
Since (A) is nonincreasing, (87) yields T(R) = T(Car) = Cu — Cum, 
completing the proof of part (v). 


IV 


APPENDIX C 
Source with Memory 


In this appendix, we show how to modify our definitions and re- 
sults for a source with memory. We will take the source output 
sequence {Sz} to be a stationary, ergodic sequence (where S; takes 
values in $) with entropy (as defined in Ref. 1, Section 3.5) of Hs. As 
in Section II, we continue to assume that |S| < ©, and that the 
source statistics are known. 

The channels Qy and Qw remain as in Section II, as does the defini- 
tion of an encoder-decoder with parameters N and K. The definition 
of P, also remains unchanged, but a new definition for A is necessary. 
To see this, let us suppose that the source was binary, i.e., 8 = {0, 1}, 
with entropy Hs, and with H(S:) > Hg. Suppose also that the channel 
Qu is a noiseless binary channel, and that Qw has zero capacity. A 
possible encoder-decoder has K = N = 1 and takes X, = Sy. Such 
a scheme has P, = 0, but with A as defined in (7) given by 
A = H(S81) > Hs. Using (9), this would lead us to accept the pair 
[ Hs, H(S1)] as achievable, which would not be reasonable. Accord- 
ingly, we give a new definition of A. 

Let S*, Z” correspond to an encoder with parameters K, N as 
defined in Section IT. Let S*(7), Z”(7), 7 = 1, 2, ---, », correspond to 
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the v successive repetitions of the encoding process. Then define the 
equivocation at the wire-tap as 


A 


lim y= HISK(1), «++, S€(v)|Z¥(1), «++, 20) 
(88) 


1 2 Ky Ny 
in OREM 


With A as defined by (88), we define the sets ® and @ as in Section II. 
We claim that Theorem 2 remains valid. 

The proof of the converse-half of Theorem 2 given in Section IV 
goes over to the case where the source has memory with only trivial 
changes. Further, the results in Section V are all valid exactly for the 
source with memory. They yield that, if (R, d) satisfies (56), then we 
can for e > 0 arbitrary find an encoder-decoder with parameters J, 
K, and P, which satisfies 

KHs 


R—« (89a) 
PS se (89b) 
pH (SK |Z") 2 Hid (89c) 


Further, we can do this for arbitrarily large K. We show below that 
there exists a function f(K), K = 1, 2, ---, such that for any code 
with parameters K, NV 


4 = lim 7 HS 2) = EH(S*|Z) — f(K), (90) 


where limx... f(K) = 0, and f(K) depends only on the source statistics. 
Combining (90) with (89c), we have 


Az2d—e-— f(K). 


Since f(K) —0, we conclude that (R, d) is achievable. This is the 
direct half of Theorem 2. It remains to verify (90). 

First, imagine that the encoder-decoder begins operation infinitely 
far in the past. Let [S(7), Z(7)_] be the (S*, Z*) corresponding to the 
jth encoding operation, —«© < 7 < «. Thus, S*’ = (81, ---, Sx,) 
= LS(1), a, S(¥)] and Z*” = [Z(1), eames? Z(v)], y= 1, 2, +--+, Let 

*=[---,Z(—1), Z(0), Z(41), --- ]. Of course, 


H(S®*|Z") > H(S*|Z*). (91) 


WIRE-TAP CHANNEL ~~ 1385 


Further, 
H(S*’|Z*) = H[S(1), ---, S()|Z2*] 


. H[S(j)|Z*, S(Gi+0), +++, SQ] 


aa s H[S(1)|Z*, SQ), ---, SA) 


= yHS(1)|Z*, S(2), «--, S()] = vHES(1)|Z*, SJ, (92) 


where S’ = [S(2), S(8), --- ]. Step (a) is a standard identity, step (b) 
follows from the stationarity of the sequence {S,} and the memoryless- 
ness of the channel Qiw, and step (c) follows from the fact that 


conditioning decreases entropy. Now, let 


Thus, (91) and (92) become 
2 A (gk|Z¥) > LH(S|Z z', 8’) 
Kv = aa 


Se [H(SZ|Z’'S) — H(Z|Z’S’)] 


A 


K 
ao x LA(S|S’) + H@Z|S) — H(2|2'S’)] 


> x LH(S|s’) + H(Z|S) — H(Z)]. 


1 FA(S|Z'S’) + H(Z|SZ’S’) — H(Z|Z’S’)] 


(98) 


Step (a) follows from the fact that Z’, S’, S and (S’, Z’), S, Z are 


Markov chains, and (4). Now 


7 H(S|8') = 2 _ BS: |S' Sera ++, Sx) 


Mr iMr 


Hs = Hs. 


1 


Py 


> 
i 


Also, 
LHS) ~ Hs Ss f(K)-0, asK-> oa. 
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(94) 


(95) 


Substituting (95) and (94) into (93), we have 


1 


x, H(Se 12") > 7 LHS) + H(Z|S) — H(Z)] — f(K) 


1 
= ZH(S|Z) — S(K), 
which is (90). 
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Optimum Direct Detection for Digital Fiber- 
Optic Communication Systems 


By G. J. FOSCHINI, R. D. GITLIN, and J. SALZ 
(Manuscript received January 23, 1975) 


We report on optimum direct detection of digital data signals that are 
transmitted over optical fibers. Direct detection 1s provided by a photo- 
detector whose output current is modeled as a noisy filtered Poisson stream 
of pulses. In this model, the time-varying pulse arrival rate 1s proportional 
to a linearly distorted version of the modulating signal. We show how the 
photodetector output 1s processed to derive the minimum probability-of- 
error receiver. Special attention is given to certain practical limiting cases. 

When the average energy in the response of the photodetector to an indi- 
vidual photon ts small compared to the additive thermal notse, the optimum 
detector is shown to be linear except for the use of precomputed bias terms. 
At the other extreme are the photomultiplier and the avalanche photodiode 
where the average energy in the response of the photodetector to a single 
photon ts large compared with the additive noise. In this situation, we show 
that the optimum detector estimates the photon arrival times and then uses 
these estumates in a weighted counter. In both limiting cases, the detectors 
are specialized to one-shot M-ary and synchronous multilevel pulse- 
amplitude modulated (PAM) signals with intersymbol interference. For 
PAM signaling, we demonstrate that finite system memory allows applica- 
tion of dynamic programming to provide a detector implementation whose 
computational complexity does not increase with time. 


I]. INTRODUCTION 


In recent years much attention has been focused on communication 
over optical channels.:? Most early work was concerned with the 
physics of the electromagnetic transmission phenomena associated 
with various optical media and with the devices needed to change 
electrical signals to optical ones, and vice versa. In this paper, we are 
concerned with the optimum (maximum likelihood) reception of digital 
data transmitted over the fiber-optic channel. Our work was motivated 
by the many invaluable discussions we have had with 8. D. Personick 
on this subject. 
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We shall not dwell on the quantum mechanical limitations imposed 
on the measurements of signals in the optical frequency range. Instead, 
we adopt a practical approach and assume at the outset that direct 
detection is used to convert optical energy to an electrical signal. This 
is accomplished by using a photodetector prior to any signal processing. 
Thus, we study a classical optical reception problem with the under- 
standing that the photodetector output can be examined in every 
detail so as to extract all relevant information. 

In a fiber-optic communication system, information is conveyed by 
modulating the intensity of a light source, such as a light-emitting 
diode. This is manifested in a photon stream whose arrival times form 
a Poisson process with a time-varying intensity function. The photo- 
detector output current can then be modeled as a noisy filtered 
Poisson process whose intensity function is the sum of a dispersed 
version of the modulating wave and a background dark current. Thus, 
the central problem in communication systems employing a fiber- 
optic medium is the detection of the intensity function. Bar-David® 
and Gagliardi and Karp‘ have considered the optimal reception. prob- 
lem in the absence of dispersion (intersymbol interference) and addi- 
tive thermal noise, while Personick*-7 and Messerschmitt®® have con- 
sidered linear suboptimum receivers to combat these deleterious effects. 

Section II reviews the communication theoretic model of the fiber- 
optic channel. Section III presents two simple examples that are 
intended to focus on certain system essentials and to illustrate some 
fundamental ideas involved in subsequent work. Section IV develops 
a general representation for the likelihood functional. Sections V and 
VI consider reception when the energy in the response of the photo- 
detector to an individual photon is much smaller than the thermal 
noise, while Sections VII and VIII consider the complementary situa- 
tion of large average energy per pulse-to-thermal noise. 


I. A REVIEW OF THE MATHEMATICAL MODEL 


In the past few years, a pragmatic communication theoretic model 
for data transmission over the fiber-optic channel has evolved. The 
papers by Personick*®~7:” contain an up-to-date account of this model 
as well as provide more complete references on the physical aspects of 
fiber-optic communication. For the purpose of this investigation, it 
will suffice to think of the optical modulation process as providing a 
proportionate variation in the rate of photon arrivals at the photo- 
detector. This device, of which there are several types, is a transducer 
that converts optical to electrical signals. The photodetector output 
current is illustrated in Fig. 1, and can be described as the sum of a 
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Fig. 1—Photodetection. 


filtered Poisson process 
v(t) 
I(Q= ¥ guolt - ty) (1) 


and white gaussian noise, n(¢), with spectral density No. The photon 
arrival times #1, f2, --: are a family of independent, identically dis- 
tributed, random variables, as are the positive gains gi, g2, ---. More- 
over, these two families of random variables are independent of each 
other. The pulse w(t) is square-integrable and is the convolution of 
two pulse shapes. The first pulse is the response of the photodetector 
circuitry to the generation of a single charge-carrier (i.e., an electron 
or a hole), while the second pulse is included for mathematical ex- 
pediency so as to whiten the noise at the photodetector output.’ We 
distinguish between two types of photodetectors, those that provide 
avalanche gain and those that do not. In the latter category is the 
photodiode that operates with g; = 1, 7 = 1, ---, »v and results in a 
pulse energy-to-noise ratio /w?(t)dt/No, which is typically —20 dB. 
In other words, the response of the photodetector to an individual 
photon is masked by the additive background noise. This is in contrast 
to the photomultiplier and the avalanche photodiode where the gains 
possess a (discrete) probability distribution whose mean, g, can be 
rather large and whose variance is a power (21) of the mean." For 
these devices, the average pulse energy-to-noise ratio g? fw*(t)dt/No 
can be on the order of 20 dB. 

The stochastic process v(t), which is the number of pulses generated 
at the photodetector output in the interval (0, ¢), is a Poisson process 
with intensity \(¢), and therefore 


Pr [y(d) = N] = exp (-a@) AOL (2) 


* Note that the inclusion of a reversible operation, such as a whitening filter, does 
not affect the performance of an optimum detector. 


FIBER-OPTIC COMMUNICATION = 1391 


where 


t 
A(t) = i d(t")ae. (3) 
0 
Moreover, each photon arrival time ¢, possesses the probability density 


AC) 
p(t.) = Tue al’ (4) 


where the integral is over the observation time.t 

In the digital fiber-optic communication system under discussion 
here, the positive intensity function A(t) is the information-bearing 
signal and is the average rate of electrons produced by the photodetec- 
tor. The manner in which \(t) is manifest in the received optical signal 
(the photodetector input) is through the relation 


A(t) = kL) + Ao, (5) 


where @(t) is the received optical power, k is a constant conversion 
factor, and \o is the average dark, or ambient, current in “counts” 
per second.? Thus, information is transmitted by modulating the 
optical power and must be recovered by processing the noisy photo- 
detector output, Z(t) + n(é). As a result of transmitting the optical 
signal through the fiber-guide medium, the intensity function at the 
photodetector output will be the sum of a linearly distorted version of 
the transmitter intensity and the dark current. In the sequel, \(¢) will 
be understood to mean the intensity function at the receiver. 

Statistical averages of Z(t) are found by elementary calculations. 
For example, 


ELT (t)] 


E(q) [ : AGwit = dr (6) 


and 


oo = E(g) / d(r)w2(t — r)dr, (7) 
where E(g) and E(g*) are the average and average square of the 
avalanche gain g. Higher moments can also be readily evaluated. 

A linear channel model with additive “noise” is suggested by (6) 
and (7). In such a model, the desired signal is taken to be the average 
value of I(t), namely \(¢) passed through a filter with impulse response 
E{g ]w(t). One component of the added noise can be thought of as the 
signal dependent process J(é) — E[J(é)], which has mean zero and 


t Note that the arrival times are not assumed to be ordered. 
? In free-space optical communication systems, \(t) must be regarded as having a 
noisy component. 
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variance given by (7). In addition to this noise, the gaussian noise 
must also be included before processing. While this linear model is 
a convenient approximation in some situations,*— for purposes of this 
investigation we work with the process J(¢) directly. 

Now that all the physical parameters have been defined, the optimum 
detection problem can be stated as follows: 


Given that the intensity function can assume one of M equi- 
probable positive functions \,,(t),0 StS 7, m= 1, ---, M, the 
task of the detector is to decide which one of the MW intensities has 
been transmitted after processing J(¢t) plus gaussian noise for T 
seconds. Of particular interest is the synchronous pulse-amplitude 
modulated (PAM) signal 


A(t) = pe arf(t — kT) + Xo, 


where each data bit, ax, assumes the value 0 or 1, 1/T is the data 
rate in bits/s, and f(¢) is a positive time-dispersed pulse. 

The subject of our investigation is summarized by the question: 
How should the photodetector output, Z(t) + n(t), be processed so 
as to minimize the probability of error? 


lil. A MOTIVATING SIMPLIFIED DISCRETE MODEL—TWO EXAMPLES 


To preview, in an elementary way, some ideas that are more fully 
developed in the sequel and also to serve as a motivation to the reader, 
we present a simplified version of the model discussed in the last 
section. 

In a simplified theoretical model, the time index ¢ is assumed to take 
on the discrete set of values t1, te, ---, tz, where t; = jA. Thus, instead 
of writing 


y(t) 
I(t) = ¥ guw(t = t) 
for the photodetector response to a photon stream, we write 


J 
I(t;) = yy geqnw (tj = te) J = it 2, oe) J. (8) 


In the above expression, {q,}{ can be regarded as an independent 


Bernoulli sequence with probabilitiest 


Pr {qx = 1} =», and Pr {qz, = 0} = 1 — AX, 


T For convenience, we have taken A=1, and so we have written A, and 1—), 
instead of A,A and 1—);A. 
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where we have in mind that 0 < \;, «1. Thus, g; = 1 (or 0) represents 
the arrival (or nonarrival) of a photon at time ¢,. We make the further 
simplifying assumption that w(t; — t,) = A6é;, (A a positive constant), 
where 6;, is the Kronecker delta and is nonzero only when 7 = k. 
This corresponds to assuming that the pulses w(t) and w(t — A) do 
not overlap. Within this simplified framework, the received time- 
discrete signal is of the form 


I (t;) = 9;QiA, 4 = 1, 2, ey J. (9) 


We recall that {v9 }7_, is the intensity function associated with the mth 
hypothesis. The particular intensity which is active is, of course, 
unknown at the receiver beyond the knowledge of the finite set from 
which it was chosen. The last ingredient of our model is to include the 
fact that the observation [(¢;) is noisy and is given by 


y(t;) = gqiA + 13, (10) 


where the noise samples are assumed to be gaussian, independent, and 
zero-mean and have variance No. In relation to the more accurate 
model of the previous section, o can be thought of as the standard 
deviation corresponding to 77, n(t)dt. As is well known, the optimum 
detector computes the likelihood (the a posteriori probability density 
of the received signal conditioned on each hypothesis—in this case, 
the intensity) and selects the maximum. In statistical parlance, this 
is a standard multihypothesis testing problem. We now develop the 
form of the likelihood for two different assumptions on the nature of 
generation of secondary electrons: 


(t) No avalanche gain (g; = 1). 
(272) Discrete avalanche gain (g; takes on values 1, 2, ---, G, with 
probabilities Pl, P2) °°", pq). 


In each case, we first obtain the likelihood for one observation. Owing 
to the nonoverlapping assumption on the pulses and the independent 
noise samples, the likelihood for J independent observations is given 
as a product. Our goal is to obtain a simple representation for the 
effective’ likelihood L(™ (Ay, A2, +--+, Aj} Ys, Y2, °°*, Ys), Where the 
superscript m denotes which intensity is assumed active. Given the 
received samples yi, °-:, yy, the maximum likelihood (optimum) 
receiver selects the index m* that maximizes L™ and declares that 
intensity \*” is present. We shall find that, if No is small, then the 


t “Effective”? refers to the fact that constants common to all hypotheses are 
dropped. 
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likelihood assumes an especially simple form. Specifically, in the high 
signal-to-noise ratio case, the likelihood is of the form 


J 
Lm ~ TT APB = AP YH, (11) 
j=l 


where g; = 1 if y; 2 yr (and zero otherwise). The quantity yr is a 


threshold value that we shall derive for each example. Alternatively, 
the log-likelihood is expressible as the wezghted counter 


J 
p> Gj log \™ + (1 — q,) log (1 — A¥)], (12)! 


where gq; is an estimated photon arrival process. In the complementary 
case of small signal-to-noise ratio (Vo ~~), the detector is of the 
matched-filter or correlator type. The effective likelihood in this 
case is 


J 
L™ wo DAP ys — 6, (13) 
j=1 


where c is a constant and b‘™ is a hypothesis-sensitive bias term. We 
now turn to the specific examples. 


(t) The Photodiode (No Avalanche Gain) 
The single observation y; is defined as 


Yi = Nj, with probability 1 — » 
and (14) 
y,=At+tN;, with probability X. 


We temporarily drop the subscripts dealing with time (j) and hy- 
pothesis (m) while investigating this single observation. The likelihood 
is the mixture probability density 


p(y) = (21No)-* exp |-#, [a — ) + Aexp | au =o || (15) 


Noticing the hypothesis (A) insensitivity of the first term, the effective 
likelihood becomes 


zt et rf (16) 


Lty) = — 2) + rexp | aN. 
A simple calculation shows that the two terms in (16) are equal when 


t The reader familiar with Ref. 3 might expect an additional —A term in (12). 
Owing to the simplified Bernoulli model employed above, this is not the case. How- 
ever, the more refined analysis in the sequel will include this term. 
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y = yr, where 
r 


_A , No 1 — 
ye = $+ MPlog (+=) (17) 
For small No, yr & A/2 and the graph of L(y) converges to the solid 
line shown in Fig. 2. So, for No/A?|log \| small, the effective likelihood 
can be approximated as 


deed ’ Y = YT 
Ly) = Ay A? (18) 
nexp {$2 — I, yY > yr. 
The sense of the approximation is expressed by the following easily 
proven statement: For each 6 > 0, one can find an No > 0 so that 


L(y) 

good | So = 0: 19 
ZW) ve 
To simplify the likelihood, note that exp {(Ay/No) — (A?/2No)} and 
yr are hypothesis-insensitive and can be deleted from the effective 


likelihood, and since we are assuming that \ is extremely small, 1 — 
can be treated as 1. The effective likelihood is then simply 


L™ = [Ae 35, (20) 


where g; = 1 if y; > yr and zero otherwise. Because of the indepen- 
dence of the noise samples and the nonoverlapping property of the 


n| 
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Fig. 2—Convergence of graph of Ly) to the asymptotic form (No — 0). 
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Fig. 3—Threshold-based weighted counter. 


pulses, the likelihood for J observations is the product 
J -_ 
L™ = II [ag 3, (21) 
jm 
which yields the weighted counter 
J 
log L™ = Y G; log \™ (22) 
j=l 


shown in Fig. 3. The receiver selects the index that maximizes (22) and 
declares that the corresponding intensity was transmitted. 

In the complementary case of low signal-to-noise ratio (Vo > ©), 
we expand the likelihood function in a Taylor series and retain the 
dominating terms. This step must be done with care, since the nu- 
merator of the exponent has variance No, while No also appears in the 
denominator. By normalizing the exponent, it is seen that the variance 
of the exponent is proportional to 1/No; thus, the exponent will be 
small and a series expansion is useful. Keeping the first two terms in 
such an expansion of (16) gives 


; Ay; A? 
Ly) = 1-2, (44 - 57), (23) 








and the likelihood for J observations becomes! the digital correlator 


t We have used the fact that d;/NoLAy;—(A2/2)] «1, and with «<1 that 
II(1l + 6) ~1 + Ye. 


FIBER-OPTIC COMMUNICATION 1397 


(matched-filter) 
J Ay; A? J A? 
(m) — = ee ae (m) oe Has 
in| ns ( No an) | x, (49. 2 ) oa 
which is shown in Fig. 4. 
(41) The Photomultiplier or Avalanche Photodiode (Discrete Avalanche) 
Again, we start with the single observation case but now, because 
of the avalanche mechanism, a single primary gives rise to 1 or 2 or 
--, G@ secondaries with probabilities p1, p2., ---, pe, respectively 
(9p: = 1). So the measurement y is modified as 
n, with probability 1 — » 


A+n, with probability \p1 (25) 


a : 
GA +n, with probability Ape. 
The likelihood is the mixture density 


_@—A) oe G Ap ty =A? 
ply) = BN, exp | ¥ | + >» SN exp | a) Paes (26) 


Factoring out hypothesis-insensitive terms, the effective likelihood 
becomes 








242 
lAy PA r 7) 


G 
Liy) = (1 -—A) + 20 pr exp | ee — 


As No— 0, we notice that L(y) ~ (1— A) for y < A/2. When 
y > A/2, let 1A denote the number A, 2A, ---, or GA that is closest 





(M) 1 (M) 
Aj > Aaj 


Fig. 4—Elementary version of digital correlator. 
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to y. Then the series appearing jn (27) will be dominated by one term, 
and the likelihood becomes 


Wa ONG as Nyo— 0. 


PA? 
L(y) ~ pir exp| Mv bay 
Proceeding as in the previous example, we consider both No and 
small and drop hypothesis-insensitive terms from the approximate 
likelihood to obtain 


Ey) = [a34, (28) 


where g = 1 when y = A/2 and zero otherwise.' Moreover, note that 
the threshold is the same as in the nonavalanche case. This is because 
the detector is only interested in ascertaining whether or not a photon 
has arrived and need not estimate the magnitude of the avalanche gain. 
Again, for J measurements, the corresponding log-likelihood expression 
is simply the weighted counter 


L = ¥ Glog (29) 


As No, we again expand the likelihood (27) in a Taylor series to 
obtain 


pee 2f lay _ PAS 
L(y) =1 —a{1 a a AL (30) 


which, for J measurements, becomes 


, J G 

log L(™) = Sap i= Be fi Be - Fe |]. (31) 
j=l l= No 2N 

The above is again interpreted as a correlator where \%” is correlated 


with Ay;/No- Direilp: = (Ay;/No)ELg]. 


IV. THE MAXIMUM LIKELIHOOD DETECTOR 


Here, we begin to answer the question posed at the end of Section 
II by presenting a derivation of the likelihood function associated 
with the received signal. The likelihood function is the probability 
measure of the photodetector output, given that a particular intensity 
is active. It is well known” that, when one of M equally likely signals 
Am(t) 1s transmitted, the optimum (minimum probability error) 
detector computes the M values of the likelihood function evaluated 
at the received waveform and declares that the jth signal was sent, 
where the jth likelihood function is the largest. 


T As expected when Ny — 0, the avalanche gain provides no essential benefit. A 
more interesting asymptotic evaluation and one that is more akin to reality is obtained 
by parameterizing the gain distribution such that E[g]/No- =. 
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We denote the received signal by 
y(t) =Intt) +n@), OStST, (32) 


where I(t) is the information-carrying, filtered, Poisson process 
y(t) 
In) = Z, guw(t — ty), (33) 


and where the index m [corresponding to \(é)] is hidden in the 
statistics of {¢,} and v(¢). These statistics are described by (2) to (4) 
with \(¢) replaced by \»(t). 

The task of the optimum receiver is thus to process the photodetector 
output y(t) for 7 seconds and then decide which intensity function 
Am(t), m = 1, 2,---, M is in effect. As we have mentioned earlier, 
the random variables {g.} represent the avalanche gains, and the pulse 
shape w(t) is so far arbitrary with the only requirement being finite 
energy. Although in actual practice the noise at the output of the 
photodetector is not white, it can be whitened by a filter before addi- 
tional processing and the effect of this filter will be manifest in the 
shape of w(é). 

The conditional likelihood function [when [,,(¢) is fixed] has the 
standard form 


ey ee | [" taovoa — sr f "ra(oat|- (34) 


The desired likelihood is the expectation of (84) with respect to Jn (t) 
for fixed m, Le., 


L(y) = Ex{LmLy|Im]}- (35) 


Once the intensity A(¢) is specified, the above expectation is taken 
with respect to the number of arrivals, the arrival times, and the 
avalanche gain values. The detailed evaluation of this expectation and 
the interpretation of the resulting structures, in terms of implementable 
physical operations on y(t), is our objective. The exact structure is 
sufficiently complex that many judicious approximations will have to 
be made to glean the essential nature of the operations. 

We remark that a representation of (85) in terms of an estimator- 
correlator structure has recently been treated in the literature.!?. 4-16 
The optimum detector has been shown to be a correlation detector 
and the deterministic signal in the classical correlator is replaced by its 
least-squares estimate. This is a reformulation of the detection problem 
in terms of an estimation problem. Proponents of this method have 
taken the viewpoint that various suboptimum detectors are suggested 
by this formulation. A typical approach might be to replace the least- 
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squares estimate by the linear least-squares estimate or some other 
approximation, and to approximate the resulting stochastic integral 
by conventional integrals. While this might be reasonable, it does not 
indicate the direction of the approximation. We prefer an approach 
that, to be sure, has many approximations and makes use of estimates 
in place of the true quantities, but that can be explicitly related to the 
optimum detector under the asymptotic conditions of large and small 
signal-to-noise ratio. 

Toward this end, we proceed by writing (35) in more detail. Neglect- 
ing edge effects on the integrals and assuming that the observation 
time TZ is much larger than the effective duration of a single pulse w(#), 
we can express the inner product and the square term indicated in 
(34) as 


T » 
fo InOyWat = ¥ geP (te), (36) 
where 
T 
Pays f w(t — te)y (dt. 
The square term is written as 
T v 
[) BOd =X n9RG - 13, (37) 
I= 


where R(t) = fo? w(r)w(t — r)dr is defined as the pulse correlation 
function. 
Substituting (86) and (37) into (35), we obtain 


se 1 Z 
Ln(y) = Ey; exp | Py GP (te) — IN< ee gg sh (tk — i) (38) 


Employing the vector notation g, = (91, gz,---,g,) and t,= 
(t1, te, --+, t,) gives the expression 


j pee 1 id 
Lnly) = Eye, | exp | Ne xX gkP (tx) — Ne ames gxg iF (tx, — «| , 
(39) 


and after performing the indicated expectations we obtain a detailed 
representation of the likelihood function 


C 1 T n n 
Im(y) = exp [—An(1)] | X f° dts TT dw(te) TT ou) 
n=0 1! ‘g, Jo k=1 bai 
1 n 1 nn 
X exp No ~ guP (te) — ONs ud gigi (te — t;)¢ , (40) 
where p(gi) is the (discrete) probability density function of the ava- 
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lanche gains and where it is understood that, when n = 0, the summand 
is taken to be unity. 

To more easily interpret and/or mechanize the likelihood calcula- 
tions, it will be convenient in some applications to assume that the 
photon arrivals can only occur at discrete instants of time {7A}, where 
A is some fixed (small) interval and 7 = 0,1, 2, ---, J. The integer J 
will be defined as the closest integer to T/A. This assumption is easily 
accommodated in (40) by replacing / dt, with a multidimensional 
sum )>-¢, over the lattice {¢, = jA:k = 1,2,---,n;j7 =0,1,---, J}, 
and by replacing A(t, = 7A) with the probability that' 7A — A/2 
St. S$ 7A + A/2. The likelihood function under this set of assump- 
tions then becomes 


Im(y) = exp [—Wa(7)] © AE TT Mt) TT ooo 


tr n k=1 


x exp | 5 -| & uP) - . "E sngnk(e — tm) | (41) 


which will be referred to as the (time) discrete likelihood function. 

The two infinite functional series, (40) and (41), are not of much 
use as they stand. However, under a variety of physically realistic 
situations and by making suitable physical approximations as well 
as asymptotic expansions, we shall be able to deduce from these repre- 
sentations real-time implementable signal-processing algorithms. 

By suitably normalizing the likelihood functions, (40) and (41), 
1/No can be replaced by the (pulse) signal-to-noise ratio. This param- 
eter a? will play a central role in our subsequent treatment, and its 
relative size will dictate our particular approach. The normalization 
entails replacing R(t) by R(t)/R(O), P(t.) by P(t.)/R(0)g, and the 
random variables g; by g./g, where g = Eg.; consequently, 


gR (0) 
No 


2 


and may be viewed as an average pulse signal-to-noise ratio. As we 
have discussed in the preceding section, in some applications this 
parameter is small, while in others it is large. Thus, our investigations 
in the sequel will focus on these two ranges. Additionally, different 
treatments of the likelihood ratio are also required, depending upon 
the presence or absence of avalanche gain. 

It is instructive to give a still different representation for the like- 
lihood, which will be found useful in the sequel. Towards this end, we 
introduce a zero-mean, stationary gaussian process x(t) with correlation 


t This probability is given by 
jA+A/2 : 
i, A(i)dt = A(jA)-A. 
ja—A/2 
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function, 


Elx(@xtt + 7)] = R(r), 


and can then write (39) in the form 


Ln (y) = Hives ss | exp {# = mPa) | i, €Xp | ie =. axe) | ) 
(42) 


where we have used the elementary identity for gaussian processes 
exp | -2/2 : gug sR (te, — 1) = EH, exp {i Pa axel) ; 
ry os = 


Since, over the observation interval, (42) is absolutely integrable, 
the expectation with respect to x and the other random variables may 
be interchanged. By noting that 


E'tn.en.y XP | Pa are) = exp (—Am) 


T 
few p(Z)Am(t) exp Cia) at}, (43) 


we can write (42) in the form 
a 
Im(y) = exp (An) Ba fox (Ef p(e)Anld 
X exp [e’9;P(f) + tagjx(t) xt) (44) 


In particular, in the absence of avalanche gain, i.e., p(g) = 6(g — 1) 
(44) assumes the compact form 


T 
Ln(y) = exp (—An)E, [exp @) Am(t) exp [e?P(t) + ia () at) 
(45) 


It may appear that the introduction of the process x(¢) did not 
simplify matters, since the explicit evaluation of the expectations 
again leads to an infinite functional series without adding insight into 
the nature of the processor. We shall nevertheless find this representa- 
tion useful. As will be seen, when suitable approximations are made and 
asymptotic behaviors explored, a great deal of insight can be gained 
from the alternative representations for the likelihood’ (40), (44), and 
(45), as well as the discrete likelihood (41). 


Tt By normalizing the exponent, i.e., introducing «2, we should actually use new 
symbols to denote gn/g and R/R(0). To avoid introducing extra notation, we retain 
the symbols gm and R(0), but we realize that, whenever a? is present, these variables 
have been normalized. 
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V. SMALL SIGNAL-TO-NOISE RATIO (a? —> 0) 

Here we consider the physical situation corresponding to small s/n 
(a? — 0). This occurs when a photodiode is used for direct detection. 
In this application, the response to an individual photon is masked by 
the background noise, and we do not expect the receiver to make 
explicit use of the information supplied by an individual pulse. Rather, 
the aggregate effect will be important. This is in contrast to the 
“counting” receivers (for large a”), where individual counts contribute 
explicitly to the final decision. Since the avalanche gains are unity in 
this application, the likelihood function takes the form of (45). Two 
signaling situations of interest are examined next. 

5.1 M-ary signaling 


Since a? <1 (typically, a = —20 dB), our approach will be to 
expand (45) in a power series in a? and retain the first two terms.' 
Consider the following Taylor series approximation to the argument of 
the exponent in (45). Again dropping the index m, let 


T 
E(a, x) = exp { A(t) exp [e®P(t) + iat) Jt} 
0 
2 
~eh+ 20,2) + 20,2) 5. (46) 
Evaluating the derivatives, the asymptotic likelihood function becomes 


Lnly) ~ Ez E + af" Am(t)x(t)dt + 3 (f° 2Xm(t)P (t)dt 


T rT a2 st 
= i / ho (la)hw (l(b) att) “or i An (ta at | (47) 
0 0 0 
Recalling that the exponent has been normalized such that Ex = 0, 


Ex? = 1, and Ex(ti)x(t2) = R(t: — te), we get, after performing the 
averages, 


Ln(y) ~1 be | f ” w(t) P (at 


i 5 ie ie \m(t1)Am (to) R (tr — te) dtidt, — ; is rn (tat (48) 
or 
a T 
Lm(y) = log Lm(y) ~ [ dm()P (Wat 
1 T T 
<5 fo fo rwldrn(RG = te)dhdts — Hm (49) 


The detector involves linear operations on the filtered received signal 
P(t), addition of constants, and a maximization. As shown in Fig 5, 


T Of course, the same answer would be obtained by working with (40). 


1404. THE BELL SYSTEM TECHNICAL JOURNAL, OCTOBER 1975 





TT 
Lm =1/2 ff Am (ty) Am (tg) R(ty to )dtydty 
00 


+A, 
(m = 1,2,- ; M) 


Fig. 5—Correlator filter for M-ary signaling. 


a realization of the receiver is obtained by first passing the incoming 
signal, y(¢), through a filter with impulse response w(—t)/R(0) to pro- 
duce P(t). This signal is then passed through a bank of M filters with 
impulse responses An(Z — t), m=1,2,---,M and sampled at 
t = 7. This is the first term in (49). The other two terms are precom- 
putable biases. The detector then chooses the index m*, which achieves 
the max L,,(y), and the corresponding \m*(t) is declared to be the 


transmitted intensity. 

There is a pleasing interpretation of this receiver which is reminiscent 
of the ‘‘linear’’ model discussed in Section II. If one were to consider 
the detection problem when the signal J(¢), given by (1), is replaced 
by its average E[I(t)] = I(2), given by (6), then the optimum detec- 
tor in gaussian noise would base its decision on the likelihood function 


gz i "(OE (dt — i "CE () Pat. (50) 
Substituting (6) into (50) gives 
g = [ave [owe ~ t)(r)dr 
5 * | 7 [ * we — WAU wd — trCdhidt, | a 


= f “XCP Gare 5 i z - ” (ts) (ts) Rta — ts) dtd. (51) 


FIBER-OPTIC COMMUNICATION 1405 


Note that (51) differs from (49) only by the bias term Am, the prob- 
ability that no photons have arrived at the photodetector. We con- 
clude, therefore, that the optimum detector structure in the case of 
small a? is thus ‘‘matched” to the average signal. 


5.2 Optimum detection of PAM signals via the Viterbi algorithm 


We will now develop the optimum receiver structure (still for small 
a) when the intensity is a pulse-amplitude modulated (pam) signal 


A(t) = = aflt—nT), OK<t<T (52) 


where each a, can assume the binary values 0 or 1, f(t) is a positive- 
valued pulse that incorporates the distortion of the optical medium, 
1/T is the symbol rate, and 7 > kT. Note that in writing (52) we 
have dropped the subscript m which we have used to identify the trans- 
mitted signal (intensity), since for PAM signaling it is generally more 
convenient to think of the receiver as finding that sequence {a,} which 
maximizes the likelihood. Substituting (52) in (49) and emphasizing 
that the likelihood function is now to be regarded as a function of a 
particular data sequence (which uniquely corresponds to a specific 
intensity) gives 


5 k k 
L(a1, da, +++, ay) = a Anen — : ae Opi Kie nis (53) 


where 


oo2 I TP) — BIE — nT (54) 


is the response at time nT of a filter matched to f(¢) when the input 
is P(t) — 3, and the correlation-type function & is defined by 


Kn-m = [ra (fase — nP)w(t — )) 


x (fo ase —mT)w(t — *)) 
= i U(r — nT)U(r — mT)dt 


= f° U@ULr — (n= mT IA, (65) 


Tt Note that we have neglected the dark current Ao. This obviously does not alter 
the final results. Also, the results can, in a straightforward manner, be extended to 
the multilevel case. 
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with 
T 
Gea ny S i Gi Sane = bak, 


and the observation time 7 is taken to be extremely large (TJ > T). 

The receiver structure indicated by (53) to (55) is similar to the 
maximum likelihood (mL) receiver for detecting a PAM signal distorted 
by a noisy linear channel.!” The received signal is first passed through 
the matched filter w(—t), and then (minus the bias term 4) matched 
to f(—t). The result is sampled at the synchronous instants n7'. This 
produces the set of sufficient statistics {z,}, from which the hypothesis- 
insensitive bias term 4 7% m=1 2. @ndmKn—m is subtracted to produce the 
likelihood function. 

The method by which the likelihood (53) is sequentially maximized 
in real time has become known as the Viterbi algorithm (va), as a 
result of its application to the analogous problem of mu detection of 
linearly distorted pam data signals. 

The va is a dynamic programming algorithm that uses the “‘finite 
memory” of Kn, i.e., the fact that there will always be a k such that, 
for all practical purposes, 


Kr = 0, |n| > k. (56) 


Because of (56), it is easy to see that the likelihood, (53), can be written 
in the recursive form 


k 
D(ay, dz, +++, Ax) = Lay, dz, +++, Ax—1) + axe, — Fax yy : Kiem (Ot) 


By introducing the sequence of state vectors {o,}, where 

On = (An—(-1), ++ *) Gn), S12) ek; (58) 
the likelihood can be written in the form 

L(o1, +--+, on) = Lo, ---, ox-1) + h(Zxj ox). (59) 


As is well known, the maximization of the function L(o, ---, o,) with 
respect to its arguments is amenable to solution via dynamic program- 
ming since (59) is satisfied. Since this is the case, the optimum receiver 
now assumes the structure shown in Fig. 6. 

In summary, it has been shown that the mt receiver for the limiting 
case of small s/n has a structure that is asymptotically approximated 
by the receiver designed to detect a known signal in gaussian noise 
(with the inclusion of certain precomputed bias terms). We remark at 
this point that the application of the Viterbi algorithm is, of course, 
only productive when intersymbol interference is the dominant im- 
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Fig. 6—Optimum detector (large noise) for PAM signaling. 


pairment. In the context of the above discussion, this will be manifested 
in the values of ®, for n ¥ 0. These values depend on the data rate 
relative to the channel dispersion. As in data transmission over voice- 
band channels, other methods of processing such as linear and decision 
feedback equalization should provide good results so long as the inter- 
symbol interference is not inordinately large. It is clear from (53) 
that when the distortion is small enough so that the quadratic term 
can be neglected, the optimization of the likelihood with respect to 
the data symbols can be carried out on a term-by-term or bit-by-bit 
basis. In other words, passing z, through a slicer provides optimum 
detection. As the distortion becomes more severe, the quadratic term 
appearing in (53) must be retained. The linear receivers reported by 
Personick>—? and Messerschmitt?® can be obtained from (53) by 
differentiating this expression with respect to the data symbols and 
then quantizing the result to the legitimate transmitted data levels. 
As the distortion increases still further, it becomes necessary to maxi- 
mize (53), as it stands, via the Viterbi algorithm. Selecting one of these 
detectors in any given situation requires an evaluation of the error 
probability to quantify the effect of distortion on the system per- 
formance. 


Vil. PERFORMANCE ANALYSIS OF THE OPTIMUM DETECTOR FOR BINARY 
ONE-SHOT SIGNALING 


6.1 An upper bound on the error rate (a simple example) 


Having a description of the optimum detector structure for a? — 0, 
it is interesting to inquire how well it performs in certain signaling 
situations. Unfortunately, the M-ary mode of operation is extremely 
difficult to analyze, and even the general binary case poses insur- 
mountable mathematical difficulties. We have, however, been able to 
analyze several special cases of interest that provide insight as to the 
effect of various.system parameters on performance. 

In the binary signaling case, information is conveyed by sending 
either intensity \1(¢) or \2(é) with equal probability. From (51), the 
ML detector has the realization shown in Fig. 7. The detector, in this 
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Fig. 7—Optimum detector for binary signaling (a? — 0). 


situation, computes the statistic 


T 1 rr 
we i [a() — oIPWd — 5 fa — Jae 


1 T = 
-5f [RO DEMON) — Ora) Teer, (60) 


and p is then compared to zero. When yu > 0, it is decided that \1(é) 
was sent, and when yp S 0, A2(é) is chosen. In (60), the indicated 
quantities are normalized such that 


PO = ay [yw - dar 


and R = R/R(0). 
The probability of error is 


Pe= 7Pr[(wz0ly) =h +n),0 st 


Sts] 
+3Prily < Oly@® = 1.7) +n(),0 St 


< 
ST], (61) 


where 
L® = w=) with Ely] = 7 r(E)dE, 
and where 


I2(t) = » w(t == $4) with [vy] = i: ho(E)dé. 


It turns out that the evaluation of (61) is not mathematically tractable 
when A; and )2 are arbitrary positive time functions. Even reasonable 
bounds on (61) are difficult to calculate in general. However, for con- 
stant intensities, exponentially tight upper bounds can be obtained. 
While the restriction to constant intensities might appear severe, it 
is shown in the appendix that in the absence of both dark current 
and gaussian noise the optimum choice of signals will have one intensity 
equal to zero while the other is arbitrary and need only satisfy a power 
constraint. Here we wish to illustrate a bounding approach for one 
special case where the upper bound can be obtained in closed form. We 
analyze the error rate for a system slightly modified from that depicted 
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1/24 Ay 02 f fre —t’) dt dt’ 
00°08 





Fig. 8—Optimal detector for a? small and A: = 0 and A2 = A. 


in Fig. 8 for \y = Oand Az = X. The modification will involve adjusting 
the threshold’ so that our upper estimate of the probability of error 
when ), is sent is equal to the estimate of probability of an error in 
the complementary situation. 

In the binary system under consideration, the information symbol 
1 is encoded into the intensity function A\,(t) = 4,0 St S 7 and the 
information symbol 0 into the intensity \2.(t) = 0,0 St S T. Notice 
that the dark current is assumed to be zero. The detector structure we 
wish to analyze is depicted in Fig. 9. Here, the information-bearing 
Poisson process is passed through a matched filter w(—t)/R(0), then 
integrated, and the result compared with a threshold set at F. If yp 
(refer to the block diagram) exceeds F,, the symbol 1 is chosen and if 
uh S&S F, the symbol 0 is chosen. Our chief interest in this example is to 
exhibit the interplay between the various parameters in this extremely 
simple but informative situation. 

As seen in the diagram, 


p= vf" Rat [" [° nowt - 2atar (62) 


or, equivalently, the test statistic may be written as 
wo = v + Nd, (63) 


which is compared to a threshold. Note that uo is just a scaled version 
of uw, and no is a zero-mean gaussian random variable with 


te [Re — 1r)dtdr 


| [7 Roa 


Observe that, in this situation, the receiver is Just a counter since the 
test statistic represents the total number of photon counts observed 
in the entire observation interval plus an added gaussian random 
variable. 


E{ng} = No 


> 
q 
we 


(64) 


tT By the threshold, we mean the bias terms appearing in (60), ie., the last two 
terms. 
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Fig. 9—Detector for a? small and \1 = 0, Az = A with threshold modified so error 
probabilities are equal. 


The integer random variable v is Poisson-distributed with 


Efvy]=ar when 1 is sent (H) 
and (65) 
Ely] = 0 when 0 is sent (Ho), 


where Hy, and H, are symbols distinguishing the two situations. The 
probability of error is then explicitly given by 


Pe=3Pr[w> F|Ho]+3Pr(h Ss Fl As, (66) 


where we have made the assumption that Os and Is are transmitted 
with equal probability. 

Since (66) cannot be expressed in closed form, we seek an expo- 
nentially tight upper bound. Applying the Chernoff bounding tech- 
nique, we notice that the error rate under the null hypothesis, Ho, can 
be upper bounded immediately since under this hypothesis v = 0. 
Applying the bound yields 


2 
P= Pr[u > Flite] 5 exp |~ Fa} (67) 
The second term in (66) can likewise be upper bounded since the 


moment generating function of » under H, is known. This procedure 
gives 


2 2 
P, = Pr[yw S F|HA,] S exp {or + S| atam(—o), 6= 0, (68) 


where 
My\n,(9) = Efer*| Hi} = exp [AT(e? — 1)] 


for 6 = 0. The bound (68) then becomes 
a"? 
P, S exp {or + SF 4 are — Df, 6> 0, (69) 
and it is optimized by finding a 6* such that 


H(6*, F) = min {or + a + AT(e-8 — }- (70) 
0>0 
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To make the upper bounds on P, and P2 equal, we select an F = Fy 
such that 
E(6*, Fo) = Fi 
Re ge 
This, then, yields the final upper bound on the error rate 
P. S exp (—F3/20?). (71) 


By differentiating (70), we see that for a positive solution to exist it is 
required that 0 < F < XT. Unfortunately, such a solution cannot be 
obtained in closed form. However, lower bounding 1 — e~* by 6 — 6/2, 
which in turn upper bounds (69), we find that 


\T — F 
Ass 
6 AT > 0, (72) 
and consequently 
2 
Pi S exp {—6*(Av — F) + > (Ar +0), (73) 


where 6* has been chosen to provide the tightest bound. 
Having 6*, the threshold Fy is obtained from 


FR_ (AT — Fo)? 


o AT so" 
Solving this quadratic equation and selecting the only reasonable root 
for Fo give 
Fo = —o% + vot + @PAT. (74) 
Substituting (74) into (72), the bound on the error rate finally becomes 
P.<em|-$ W140 - vor], (75) 


Average Noise Power 
= ‘ae 2 lO 
WAC ey ans eel oe Average Shot Noise Power 


It is instructive to express the bound (75) in the following alterna- 

tive form 
Ps < en Pf le), (76) 

where 

RK? 
K + o? 
2 Average (signal)? 
~ Average Total Noise Power 


and where f(c) = [1 +ec—-—ve&+cf. 


As can be checked, f(c) is a monotonically decreasing function of c, 





> 
I 
Nol 
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and has the properties 
lim f(c) = 1 
c>0 


lim f(c) = 


c7o 


ales 


Thus, P. S$ ef) + e-9, asc 0. 
This is the situation that prevails when the shot noise dominates. 
On the other hand, 


Poe eee, as C0, 
which is the situation when the gaussian noise dominates. 


6.2 Implications of the error bound 


The first observation concerning (75) is that, as C—0, P, 
< exp {—K/2}. This can be achieved by making o? — 0. This implies 
that either the gaussian noise is zero or that the number of counts is 
very large. However, in the absence of gaussian noise (as well as dark 
current), it is clear that the only way to make an error is when there 
are not any counts (v = 0) under H;. The chance that » = 0 under 
H, is just exp {—K}. In the absence of gaussian noise, this is clearly 
the very best performance one can hope for. Notice that the upper 
bound predicts an outcome which is 3-dB poorer than this ideal. The 
factor of 2 in the exponent of (75) is attributed to our bounding tech- 
nique. What, in fact, happens as o? — 0 is that 6* increases, and that 
the lower bound @ — 6?/2 becomes loose, the upshot being the factor 
of 2 in the exponent. To see that this factor of 2 is indeed a quirk of the 
parabolic approximation to the exponential, consider the exponent in 
(69) as o? > 0. It is clear that the optimum threshold and @ are, re- 
spectively, zero and infinity, which when substituted in (69) does 
indeed give eT (K = AT). 

Another aspect of the bound, however, is that ideal performance 
can be achieved with this detector structure (which is optimum for 
o?—>o, the large gaussian noise situation) when the noise vanishes 
(o? > 0). This suggests that for the case of constant intensities the 
linear threshold detector is robust, i.e., it performs well over the entire 
range of o* (or a’). 

We now use the error bound to determine the number of counts 
required, for reasonable operating physical parameters, to achieve a 
desired error rate. Note that, from (64), after a simplifying calcula- 
tion on the double integral, we obtain 


T 
tR (t)dt 
r i 
= m2 (14 -f tR (tat) a ee 
0 


=, 5. 407) 
A a [Prod 
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where 
T 
Ae i R(tdt. 
0 


Introducing the pulse stretch factor, 





T T 
S= I tR(t)dt vi I R()dt < f, (78) 
0 0 
into (78) and recalling that a? = R(0)/No yields explicitly 
21—r R(0)S 
oan see NS 
7 oe fr A’ (79) 


where 0 < r = S/T <1. What, then, can be said about the choice 
of the parameter r? Can it be selected at will? Within a good approxi- 
mation, SR(0)/A ~ 1. Clearly, the best choice of r appears to be 
unity, since r = 1 reduces the noise variance to zero. Recall, however, 
that, when the mathematical model was initially introduced, it was 
tacitly assumed that the observation interval was much larger than 
the width of the pulses emanating from the photodetector so that edge 
effects could be neglected. This alone would restrict the range of r to 
be no more than, say, 0.1, which would indicate that r does not appear 
to be an independent parameter. With r = 0.1, we may conclude from 
(79) that the effective gaussian variance of the scaled system is roughly 


o? = 20/a?. (80) 


Returning now to (75), we see that ideal performance is achieved 
when 


and when (80) is substituted in the above, we arrive at the condition 
that 

20 

re «K<1-— Ka’? > 20. (81) 
As an example, let a? = 1/400, which, according to 8. Personick,' is 
a reasonable number for this parameter. This implies that K > 8000 
is required to achieve ideal performance (i.e., the error rate in this 
range approaches zero like e~*). On the other hand, suppose it is 
desired that P. < 10~*. This would imply that 


K 
2 





{V¥(20/Ke®) + 1 — V¥20/Ko?}? ~ 20. 
t Private communication. 
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For instance, a? ~ 1/400 implies that K is on the order of 1200. The 
above discussion quantifies the facts that to achieve good performance 
the total number of counts must be large or, if the background gaussian 
noise is small then fewer counts are needed to provide satisfactory 
performance. 


6.3 Some conclusions concerning optimum detection for constant intensities 


Note that the linear receiver, which is optimum when a? — 0, seems 
to be robust—at least for binary systems signaling with constant 
intensities. The optimum detector in the small s/n case (a? — 0) yields 
a decision variable based on the total number of observed counts as 
evidenced from (63). Of course, for the error probability bound to be 
tight, the average number of counts, K, must be large enough that 
o?/K <1. On the other hand, we saw that the optimum detector 
structure in the case of large s/n (a? — ©) combined with narrow pulses! 
(r < 1) is also a counter. The only difference is that the counts in the 
a? — 0 detector are linearly corrupted by gaussian noise, while the 
counts in the a®—>~« detector are determined by quantizing the in- 
coming signal to the nearest integer in the presence of the added 
gaussian noise. The latter operation is, of course, nonlinear. Never- 
theless, when the added noise is small (a2), the two operations 
are approximately equivalent, thus explaining the robustness of the 
linear receiver and the results of our theory. 


Vil. LARGE SIGNAL-TO-NOISE RATIO (a?—> ©) AND NARROW PULSES 


When a photomultiplier or avalanche photodiode is used to provide 
direct detection, the parameter a? is much larger than unity. In this 
application, the response of the photodetector to a single electron or 
hole is much larger than the background gaussian noise. In this situa- 
tion, intuition dictates that the detector make use of the ‘‘estimated”’ 
arrival times of the individual photons. Here we discuss a special case 
that will bring out the essential structure of the optimum detector. 
The situation examined is when there is no avalanche gain and the 
individual pulses w(t) are time-limited to an interval much smaller 
than the observation interval. The more general situation is treated 
in Section VIII. 

The approach taken in this section is to use the gaussian process 
formulation (45) and attempt to approximate the indicated expecta- 
tion with respect to the x(t) process. For this approach to be productive, 
we must assume that R(t) has effective duration A. We may then 


t This was demonstrated in the examples of Section III and is reestablished in 
Section VII. 
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approximate the integral appearing in (45) by a discrete sum, i.e., 
T J 

/ dt exp [eP(t) + tax(t) ]An(t) ~ A XY exp (a2P; + tax;)Xm(jA), (82) 
0 j=l 


where P; = P(jA) and x; = x(jA). 

The implication of (82) can be viewed in several ways. Of course, 
as A—0O and J —~o, irrespective of the correlation function R(t), 
the discrete sum is an excellent approximation to the integral. But 
sampling the integrand at the rate 1/A does not necessarily guarantee 
that the sum is a good approximation to the integral. Yet to derive 
any utility from representation (45), we must sample at a rate 1/A 
so that the sequence of random variables {x;} can be regarded as 
identically and independently distributed. Unfortunately, this is the 
only case for which we can compute the indicated averages in a useful 
form. What then do we mean by (82)? To make sense of this repre- 
sentation, we must reinterpret the distribution of the arrival times, 
{t.}. Evidently, the reason we have an integral representation instead 
of a sum is because we have assumed that the arrival times obey a 
continuous distribution. However, if we assume at the outset that 
the arrival times {¢,} can occur only ata set of discrete points {t, = nd}, 
then (45) will contain a sum instead of an integral. This procedure is 
equivalent to that used to obtain (41) as the discrete version of (40). 
Hence, a rigorous interpretation of (45) is that the Poisson arrival 
times can only occur at the discrete instants of time {jA}, 7 = 0, 1, 2, 

. If we now assume that the quantization of the arrival times to 
units of A is such that R(A) ~ 0, then the set of random variables 
{z;}7_, are mutually independent. Exploring this line of reasoning, 
(45) can be written as 


L(y) = TI Balexp {Adsexp @P; + iaz)}], (83) 


where \; = A(jA), and we have suppressed the index m denoting the 
particular hypothesis being tested. 

Expanding (83) in a power series and carrying out the indicated 
expectation give 


eLy) = TE (EO exp fat (nPi-F)}) 


We are now in a position to exploit the assumed large value of a’. 
In other words, we are interested in determining the behavior of (84) 
as a? — «©. Towards this goal, consider the sum 





= 5 ON" exp fatinP; — nt/2]}. (85) 


n=0 
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This series is in the form 


: Bn exp (on); (86) 


where 6, and y, have the obvious identifications. 
Let 7; be the largest of the y, and 6; be the corresponding value of 
Bn. Then (86) becomes 
Byexp (9) ( St exp [at(rn — 791), ($7) 
Ls J 
where each yn, — ¥; 1s negative. Since (86) converges absolutely, the 
infinite sum can be rearranged in such a way that the exponents are 
decreasing; thus, the rearranged sum is recognized to be a Dirichlet 
series! in the parameter a. From the elementary properties of such 
series, we deduce that, except for the n = 7 term, the summation por- 
tion of (87) converges to zero as a2 +0. So, as a2 >, (87) behaves 
like 
8; ~ Bj exp (e745). (88) 


Now returning to the series in (85), we let n,; denote the strictly 
nonnegative integer attaining 


2 
max enP; — * a? ‘ (89) 
n€{0,1,2,---} 2 


i.e.,n; = LP], where [P | denotes the nonnegative integer nearest to P. 


The corresponding coefficient becomes 
5 Ani(d;)% 
B= a 


(90) 


Thus, as a>, 
L(y) = log L(y) = —A + log 
J nj A) nj 
x {x (SRO exp Lat (nsPs — n8/2))|- oD 


Discarding the hypothesis-insensitive terms, (91) can be rewritten 
in the form 


fe: J 
Linly) ~ —Am + 2 n; log \%, (92) 
a 


where we recall that »; = 0 whenever P; < 4. Note that (92) is 
similar to the detector described by (56); however, the different 
statistical model (Bernoulli as opposed to Poisson occurrences) 
accounts for the bias term —A,, appearing in (92). 

The detector structure exhibited in (92) has a simple interpretation 
and is similar to that depicted in Fig. 3. As shown in Fig. 10, the in- 
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j QUANTIZER 
v(t) | w(—t) j 
R (0) 


1/2 3/2 5/2 





Fig. 10—Weighted counter with quantizer. 


coming signal y(¢), having been filtered by w(—t)/R(O), is sampled 
every A seconds. This is followed by quantizing the samples, P;, to the 
nearest positive integer (including zero whenever P; < 3). The 
quantized samples are multiplied by the locally stored numbers log \{” 
and the results summed. The sum is added to A,, to form the decision 
statistic. Since the added gaussian noise is assumed to be small and the 
pulse w(t) is assumed to be narrow, most of the time the nearest integer 
at any time ¢; = jA will be either 1 or 0, depending on whether 
P;>4 or P; S 3, i., whether the receiver determines a pulse is 
present or absent. Consequently, the optimum detector structure may 
be viewed as a weighted counter, where the decision as to which in- 
tensity was transmitted is based on selecting the largest of the weighted 
pulse counts. 

We recognize that from an implementation point of view even this 
seemingly simple structure may pose practical difficulties. The indi- 
cated sampling may be difficult to carry out at this high frequency. 
While this is indicated mathematically, in practice the peaks of the 
signal at the photodetector output could be used to approximate the 
photon arrival times and, hence, the interrogation times. 


VII. MAXIMUM LIKELIHOOD RECEIVER FOR LARGE SIGNAL-TO-NOISE 
RATIO (a?—> ~) 

This section extends the results of the last section by indicating a 
general approach to the extremely complex problem of performing 
optimum detection when the pulses w(¢) are not restricted in width or 
shape and when avalanche gain is provided. In the presence of ava- 
lanche gain, the average signal-to-noise ratio, a?, is large. This implies 
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that the photon arrival times can be accurately estimated, and these 
estimates can then be used to aid the detector in making accurate 
decisions. One objective of this section is to indicate how the optimum 
detector estimates the arrival times. Heuristically, the receiver at- 
tempts to ‘‘whiten”’ or peak up the pulse w(t). The presence of gaussian 
noise, however small, prevents pulse whitening via linear filtering. 
The nonlinear manner in which the receiver estimates the arrival 
times is of independent interest and will be presented in the sequel. 

We begin with the most general form of the likelihood function (40). 
While the infinite functional series appearing in (40) is quite intimidat- 
ing, it has already been shown to reduce to physically interpretable 
receivers in the following special cases: (2) small signal-to-noise ratio 
(a? + 0) and (72) large signal-to-noise ratio (a2 ©) combined with 
an extremely small decorrelation time! for R(é). 

Since large a? is a practical operating condition (photomultiplier 
and the avalanche photodiode), we are motivated to examine the 
salient features of the optimal processor under these circumstances. 
We also specialize our development to the paM-Poisson intensity, or 
data signal, 


N 
Am(t) = A+ DY aft — nT), 0<tSs 7, 
n=0 


where f(t) is a known pulse shape determined by the distortion (inter- 
symbol interference) in the optical fiber and Xo is again the ambient or 
“dark” current. Here, the optimum receiver maximizes the likelihood 
function with respect to the data sequence {a{”}#_>. As it stands, the 
likelihood (40) is similar? in form to the Volterra kernel description of 
a general time-varying nonlinear functional. However, such generality 
seems to preclude any practical value, and furthermore reveals little 
of the receiver’s essence. To obtain a good approximation to the struc- 
ture of the receiver when a? —~, it will again be necessary to dis- 
cretize the photon arrival times. 


8.1 The asymptotic (a — ©) likelihood function 


In this section, the basic idea is to asymptotically evaluate the 
multidimensional sums or integrals. Note that, when a?—, the 
2n-fold integrals appearing in the likelihood become increasingly sensi- 
tive to the value of the exponent, and in the limit the integral is com- 


Tt Note that, as R(t) — 6(t), the gaussian noise becomes transparent to the receiver 
(since the integrated received signal would be discontinuous whenever an impulse 
arrived). The receiver then assumes the form of a counter. 

? The difference is that, in our application, the input P(f) is exponentiated rather 
than appearing directly. 
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pletely determined by the coordinates that maximize the exponent. 
This statement is made precise by the multidimensional version of 
Laplace’s theorem! which, apart from certain hypothesis-insensitive 
terms, gives for each n 


lim ; "ats 5 ae p(gs) TE Me) 


ae 


X exp [ [é GmP (tm) — - ms 


m 1 


ingtB(tm ~ 4) |] ~ TE oN 
x exp fet [ PG) - FEE dain —m]}, (3) 


where {ti, t, ---, ta} and {gi, 93, ---, gz} maximize the exponent, 
n 1 n 
u GmP (tm) — pee JmgxR (tm — tr), 


under the constraint that 0 St; < 7,7 = 1, 2, ---,n. The determina- 
tion of the extremizing sets appears very difficult. For example, without 
avalanche gain (i.e., gm = 1) and n = 1, it is clear that ¢j is taken at 
the point where the observable P(t) is a maximum. For example, 
when n = 2 the exponent becomes 


P(t) + P(t2) — R(t: — te), 


and the choice of ¢; and ¢, is not apparent. The values of ¢{ and #2 tend 
to be near the peaks of P(t), but this is not always the case.* The best 
choice of t; and ¢t, will, of course, depend on the interaction of the 
random process P(t) and the correlation function R(t). The problem 
of finding the set of points {¢]} is in some sense equivalent to whitening 
or peaking up the pulse w(t) in a nonlinear manner to minimize the 
noise enhancement concomitant with such an operation. Putting 


aside for the moment the difficulty of determining {fj, ---, é,} and 
{91 °**; Jn}, We can use these values to rewrite the right-hand side of 
(93) as 


[ Bocas) exp [ot | dPGd — FEE aatee, — | | 
j=l m=l1 m,k= 
47,(7) exp [a°B.(T)], (94) 


Tit a been assumed that there is only one set of variables t = (1, ---, tn) and 
++, 9n), which maximize the exponent. If there are several shi t? and g"*, 
then the right-hand side of (93) would consist of a sum of these terms. We do not 
pursue this approach, since the resulting structure is hopelessly complicated and 
appears to be impractical. 
* This would be the case whenever P(t) has equivalued maxima spaced at least a 
decorrelation time apart. 
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where we have indicated the dependence of both the coefficient and 
the exponent on the observation interval 7. Using (94) in (46) gives 
the Dirichlet series”! 


PS Fee = (1) -exp [a2Bn(T)]. (95) 


Asa — 0, it is well known that the Dirichlet series is dominated by the 
term with the largest exponent, 1.e., 


lim e4L ~ ynx(T) exp {a?Brx(T)}, (96) 


are 


where 8,* is the largest exponent.' It is evident that n* is an estimate 
of the number of (Poisson) events occurring in the interval 7 and 
that t{, ---, ¢; are estimates of these occurrence times, while gi, «++, gz 
are estimates of the avalanche gains. This is not surprising since, as 
a? —«, the vanishingly small noise implies that these estimates will 
be quite accurate. Hence, the receiver is intimately related to the 
situation considered by Bar-David,? where the Poisson events can be 
observed directly. The distinction is that estimated arrival times and 
avalanche gains are used rather than their true values. It is important 
to realize that specific estimators have been obtained for the random 
parameters. As we show in the sequel, the simultaneous estimation and 
detection described above can be recursively implemented via dy- 
namic programming. 

Since neither the exponent in (94) nor the JJ"21p(g;) term is hy- 
pothesis-sensitive, the relevant portion of the likelihood function is 


L~ e*yae(N) = o* TE ME); (97) 


where n* is the number of time points that maximize the exponent of 
(92) (which, of course, depends on 7) and {¢;}7_, are the values of 
these time points. Note that, once the exponent is jointly optimized 
with respect to t, and gn, the estimate of the avalanche gain is not 
utilized further. This is so because the avalanche gain is a property 
of the photodetector and conveys no information concerning the 
intensity function. The asymptotic (a >) likelihood given by (97) 
is exactly Bar-David’s® likelihood formula, with the true arrival 


TIf the signal-to-noise ratio is not Jarge enough so that this is not an accurate 
approximation, then one could designate n, as the second Jargest exponent, thereby 
developing the more accurate series 


L ~ exp(—A)yn* exp (Bn*) (+2 As Yng EXP (Bra) 


n* EXP (Bn*) 
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times replaced by estimated arrival times. Note that the log-likelihood 
Lm = —An(T) + ¥ log 1 (') (98) 
i=l 


is again a weighted counter, and is similar to (98) derived in Section 
VII [where the pulses w(t) were assumed to be narrow ]. 

Two shortcomings are associated with the above approach, one is 
computational and the other involves a question of mathematical 
rigor. The first point is that implicit in the expression for the likeli- 
hood (97) is the ability to solve the formidable mathematical problem, 


n 1 n 
max DY GmP (tm) — 5 DD ImgeR (tm — tet, (99) 
Zn, tn, and n m=1 mk=1 
OStS7, i=1 ; 


in real time. We are not aware of optimization techniques capable of 
this accomplishment. The second point involves the invocation of the 
large a? assumption in a sequence of operations. Recall that this assump- 
tion was used to derive (93) and then used again to obtain (96). 
While the validity of the preceding operations can perhaps be demon- 
strated (under suitable conditions), the intractable nature of (99) 
forces us to slightly reformulate our problem. 


8.2 The optimum detector when the photon arrival times are discrete 


To proceed further and obtain a physically realizable, as well as 
meaningful, detector, we discretize the photon arrival times. Adopting 
this approach, the photon arrival times are now constrained to occur 
at the discrete instants jA, (7 = 0,1,2,---,J, where J = T/A). 
This gives rise to the discrete likelihood function (41), and eq. (98) 
then involves only sums rather than integrals. This modified expres- 
sion contains a 27 + 1 dimensional sum, which is recognized as a 
bona fide Dirichlet series. Thus, we have avoided the mathematical 
question concerning the validity of an asymptotic expansion by intro- 
ducing a mild relaxation of the physical set-up. 

Applying the asymptotic condition to the 2n + 1 variable summa- 
tion again produces the expressions (94) to (99) where it is recog- 
nized that the variables {t;} are now constrained to lie on the 
lattice, 1.e., t; = 7:A, where 7; = 1, 2, ---, J. We now show that, using 
this discrete framework, the exponent appearing in (94) can be re- 
written in a form readily amenable to maximization. Note that the 
variables éj, tj, ---, t, may be thought of either as specifying a single 
point in n-dimensional space or as specifying n points on the interval 
(0, 7). This latter viewpoint turns out to be more useful. 
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We choose the time quantization A so that the probability of more 
than one photon arrival occurring in a time interval of size A is vanish- 
ingly small* under each hypothesis \»(t). In this framework, the set 
of time points {t;} specifies n points in the interval (0, 7), and the 
exponent can be rewritten as 


n 


n * 1 
Py GAE i) = 5 = ye JmGrR (tm — tx) 


k 


J 


DD gmgrdmgikk(mA — kA), (100) 


k=1 


a 


J 

= ¥ gndnP (mA) — 5 
m= 
where 0 < ¢,, S JA and where qm is 0 or 1. A value of gm = 1 implies 
that the time point mA is “active” in the sums appearing in (100), 
while gn = 0 implies that it is not. If one chooses A to provide a 
coarser quantization of the time axis, as might be required by practical 
restrictions on the sampling rate, then it is necessary to allow qm to 
assume more (integer) values than 0 and 1. To see why this must be 
the case, recall the physical meaning? of the time points {#7}. It is then 
realistic to expect that more than one photon will have arrived in a A 
interval and consequently some t; = ¢; (fori # j). The increased range 
of dm is necessary to accommodate this situation. Realizing that no 
restriction is implied, for reasons of simplicity we assume in the sequel 
that A is chosen small enough so that gm = 0 or 1. At this point, it is 
clear that the product gmQm is inseparable in the optimization of (100). 
Note that, once the optimum values of gm and gm are determined, only 
the value of q, plays a further role in the detection procedure. With this 
in mind, we let Bm = @mGm, where Bm will range over the allowable 
values of gm aS well as zero. For convenience, we call this discrete set 
B. In the context of this new notation, the optimization problem posed 
in (99) becomes 


J J 
max > BnP(mA) — 5m  BmbrR(mA — kA), (101) 
B1,°++,B7 m=1 m,k=1 
BEB 


where it is important to realize that the maximization with respect to 
n, appearing in (99), has been removed in (101) by eliminating the 
restriction that only a predetermined number of q,.’s be nonzero. It is 
also apparent that the exponent is of the required recursive form so 
that the exponent can be maximized via the Viterbi algorithm. With 


* This probability is 1 — e — \e = A 
' The {tj} are estimates of the pulse arrival times. 
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this in mind, the likelihood function can now be written as 


J 
L~e* J] [A(jA)]4, (102) 
j=l 
and the log-likelihood again assumes the weighted-counter form 


L=-f[ "dt + = q; log [A(jA)], (103) 


which is similar to the detector described by (92) but without the 
restriction on the correlation function R(t), i.e., R(é) need not be con- 
fined to an interval A. The result embodied in (92) for nonoverlapping 
pulses can be easily derived from (101) by setting R(mA — kA) 
= 6pm. The exponent then becomes >>#-,[6,P (kA) — 3462], which is 
optimized, over the integer values of 6:, by choosing 6; to be the 
quantized version of P(kA). 

The structure of the optimum detector (103) is shown in Fig. 11, 
and is of the estimator-detector type. The arrival time indicators 
{q }241 (as well as the avalanche gains) are determined by applying 
the Viterbi algorithm to the exponent. Once these values are available, 
the likelihood is computed for each hypothesis \‘™ (t) and the maximum 
is selected. 


8.3 Optimum detection of PAM intensities 


The above methodology is now applied to the optimum detection of 
a digital (paM) data signal. The 2”*+' intensity functions in this situa- 
tion are given by 


N 
Mt)h=Ao+t LMaft—-nT), OSt¥¢fs, 
n=0 


where the effect of optical channel distortion (intersymbol inter- 
ference) is included in f(t). 

To optimally detect these signals, it is convenient to rewrite the 
original likelihood expression so that time is directly expressed in 
units of A. Bringing out this dependence, the likelihood function then 
becomes 


Ly ~ exp |- f = r(at| ia Ca(jay]y, (104) 


where the index J designates time in units of A. Note that the expo- 
nent (101) is already expressed in this form. 

It is important to emphasize that a simultaneous or two-tier real- 
time sequential optimization procedure is required to extract the ML 
estimate of the data sequence, {an}?=o. The exponent is first maximized 
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Fig. 11—Estimator detector type of weighted counter. 


with respect to the {8:}4~9, and the corresponding q; values are then 
used to maximize (104) with respect to the data symbols. The optimiza- 
tion of the exponent is identical to that occurring in ML data sequence 
estimation in the presence of intersymbol interference.!® The maximiza- 
tion of the exponent will, at random intervals,’ produce optimum 
values of {q;}, say, Go, G1, «++, Gz. At this instant, the optimization of 
the likelihood LZ, can then proceed using this new information. At 
some later instant, Gx41, Gei2, °+*,) Geen Will become available and 
attention again shifts to maximizing the likelihood L,,,. As we shall 
show, the dynamic programming algorithm which maximizes the 
coefficient (103) is quite different from the conventional Viterbi 
algorithm. In fact, the application of dynamic programming to the 
iterative’ maximization of this function illustrates the more general 
principle that dynamic programming is applicable to the iterative 
real-time ML sequence estimation of digital data that has undergone 
a wide variety of nonlinear distortion. The only requirements are 
that (2) the likelihood possesses the mathematical property of addi- 
tivity and (72) the nonlinearity is of finite memory so that the notion 
of a “‘state’”’ is meaningful. In this application, both these requirements 
are satisfied. 

To apply dynamic programming to the optimization problem ex- 
hibited in (103), we need only show that the likelihood satisfy a par- 


t Owing to the merge aspect of the Viterbi algorithm. 

t The two main virtues of dynamic programming are that (2) it is essentially a 
real-time processing scheme (although there is random signal-processing delay) and 
(iz) the number of computations is linearly proportional to time (n), as opposed to 
a straightforward evaluation that requires an exponentially growing number of 
computations. 
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ticular recursive form. To put the likelihood in this recursive form, we 
define the state vector 


5; = (akg ee, aj) j= f,f4+1, De tel, (105) 
where f is the memory (in units of A) of the dispersed pulse f(t), i.e., 
f(mA) =0, au>f, (106) 


and where 7 is the closest integer to fA/T. 

As the optimum {4,;} time instants emerge from the Viterbi al- 
gorithm in a random manner (owing to the merge mechanism), they 
are classified according to which time segment (0, NT) they belong. 
Once optimum time instants begin appearing that are active in the 
(N + 1)T time segment, those optimum q,’s which are in the NT time 
segment are available to maximize the coefficient or, equivalently, the 
likelihood. 

By substituting the pam signal into (104), the log-likelihood has 
the form 


N J N 
Lj = - 2 aul! s + du q; log (r + 2, Ani (ja — mT)) , (107) 
n= j= m= 


where J is now interpreted as the index of the latest’ merge in the 
Viterbi algorithm associated with the time interval (0, N7) and 


JA 
F, = / f(t — nT)dt. (108) 


It is important to keep in mind the fact that, once the decisions 
(1, G2, ++, Gx) are available, the iterative procedure for maximizing 
the likelihood proceeds in units of 7. The log-likelihood can be put in 
the required form by letting D = 7'/A and writing the likelihood as 


N-1 ND-D N-1 
In=—-— YVaFr,+ DL qjlog (r+ aes anf(ja — mDA)) 
n=0 7=0 m=N—j—f/D 
ND N 
fayFy+ 3 gilog(do+ > anf — mDA)): 
j=ND—D+1 m=N—j-4/D 
(109) 


It is crucial to realize that the last term in (109) only involves aj;—1-1, 
Qj—1-n+41,:--,€;; therefore, with the state vector defined by (105), 
(109) can be written as 


Ly 


I 


Ly-1 + h(qn; Sy), (110) 
where 


dv = (qnp-p+H1,---,9np)- (111) 


T In other words, the next segment of optimum q,’s will penetrate beyond the time 
instant NT. 
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Fig. 12—Two-tier dynamic programming algorithm. 


It is well known that, through the use of the recursion (110), dynamic 
programming may be applied to the maximization of Dy. 

The resulting receiver is depicted in Fig. 12, and is a two-tier dy- 
namic programming algorithm that simultaneously iterates the ex- 
ponent and the coefficient to obtain a sequential (or real-time) maxi- 
mum likelihood sequence estimate of the transmitted sequence {a,}. 
While the above detector requires sampling at a rate that could pre- 
clude practical implementation, we remark that, in the large a? en- 
vironment, a peak detector could be used to estimate the photon 
arrival times. These estimated arrival times would then be used in 
a dynamic programming algorithm to mitigate the effect of inter- 
symbol interference. 


IX. DISCUSSION 


The communication-theoretic model for the fiber-optic communica- 
tion system has proven to be quite useful. Using this model, the opti- 
mum (maximum-likelihood) receiver was exhibited under a wide va- 
riety of physical circumstances for M-ary and digital pam signaling. 
Whether or not the energy in the response of the photodetector to an 
individual photon is large or small compared to the background 
gaussian noise, the detector structure turned out to be a weighted 
counter. The details of how the weighting is carried out have been 
shown to be complex in some cases. Further investigation into system 
performance is needed before assessing whether or not such complexity 
is warranted in any particular application. For values of pulse energy- 
to-noise ratio (a?) much less than unity, the structure of the optimum 
detector can be simply instrumented in terms of analog operations on 
the photodetector output. On the other hand, when a? > 1, and with 
or without avalanche gain, we have been unable to realize the optimum 
detector without first sampling the photo-detector output many times 
per symbol interval. This procedure may impose practical limitations 
on the implementation. Since the digital operations are required solely 
to estimate the photon arrival times, it has been pointed out that 
certain suboptimum operations (such as peak detection) may be used 
to estimate these instants. The power of maximum likelihood process- 
ing can still be used to mitigate the effect of intersymbol interference. 

From a communications and information theoretic point of view, 
there remain many important and, as yet untouched, problems asso- 
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ciated with the fiber-optic channel. Sharp bounds on the performance 
of the various detectors are extremely difficult to obtain, and very 
little can be said at this time. Also, questions concerned with capacity, 
reliability, and complexity need be addressed. 
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APPENDIX 


Optimum Binary Intensities in the Absence of Gaussian Noise 


In this appendix, we determine the optimum binary intensities 
\i(¢) and \2(t) in the absence of gaussian noise. We proceed initially 
by neglecting the dark current. Of course, the optimum intensities 
must satisfy an energy constraintt 


i * ul) baa) ae ae. (112) 


Consider the performance of a system that uses the equiprobable 
intensities 


Ai(t) = = 


oS 
IIA 
IIA 


Tf. (113) 
de (t) = 0 


The only way an error can be made under (113) is when );(¢) is trans- 
mitted and no photons arrive; the probability of this event is 


Pr= ze-P, (114) 


Consider now the performance of a system that uses the arbitrary 
and equiprobable intensities \1(¢) and A2(t). The probability of error 
for this system is 

Pr = 4P, + $P.2, (115) 


where P; and P, denote the conditional error probabilities given that 
Ai(t) and A(z) are active. Let 


T 
ihe =| Nak. p= 192 (116) 
0 


t Since the intensity is proportional to the transmitted optical energy, the con- 
straint is on the average energy. 
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and let Ai be greater than Ag. It is clear that, when Aj is transmitted, 
the optimum detector must make an error when there are no photon 
arrivals. These observations provide the following sequence of lower 
bounds 

Pr 2 3Pi = oe", (117) 


and since A; + A, = P we have 


Pr = ge~ At = Ze-P = Py. (118) 


It is thus established that the intensities described by (118) minimize 
the probability of error and therefore are optimum. It is also clear that 
any system that has one of the intensities equal to zero, and the other 
arbitrary (and satisfying the power constraint), will perform equally 
as well as (118). 

The effect of dark current on the probability of error can be made 
arbitrarily small by choosing \2(¢t) = O and picking A1(¢) so that the set 
of points where \,(t) is nonzero is sufficiently small. 
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Transverse Coupling in Fiber Optics 
Part IV: Crosstalk 


By J. A. ARNAUD 
(Manuscript received March 18, 1975) 


We evaluate the crosstalk between adjacent cores in an optical fiber that 
results from electromagnetic coupling. Means of reducing it are discussed. 
We find that a 0.5-um-thick layer of silver can, in principle, reduce the 
crosstalk from —20 to —1380 dB without significant increase of the loss. 
These theoretical results are obtained for two identical single-mode dielectric 
slabs. In reality, the slabs are not rigorously identical. Longitudinal fluc- 
tuations of slab thickness reduce the crosstalk by at least 40 dB. The slab 
spacing can accordingly be reduced from, typically, 11 to 6 um for a 
constant crosstalk. If the slabs are made dissimilar with a relative differ- 
ence in thickness of 10 percent, the spacing can be reduced further, to 
approximately 1.5 times the slab thickness. For example, a 15-um spacing 
is required between single-mode dissimilar slabs if the nominal slab thick- 
ness 18 10 wm, provided scattering can be neglected. 


I. INTRODUCTION 


In multichannel communication systems, crosstalk between chan- 
nels is a problem that must be considered. Typically, the crosstalk 
should be less than —20 dB. This means that, if an optical power of 
1 mW is fed into one optical guide of a cable, no more than 10 »W 
should be transferred into the other guides. Let us assume a typical 
link length of 10 km. The crosstalk measured over a 1-km-long fiber 
should be less than —40 dB if the power transfer is proportional to 
the square of the fiber length, less than —30 dB if the power transfer 
is proportional to the fiber length, and less than —20 dB if the power 
transfer is independent of the fiber length. As we shall see, the first 
power law is applicable to identical uniform fibers, the second to 
nominally identical irregular fibers, and the third to uniform dissimilar 
fibers. 

In optical fibers, the field decays exponentially in the cladding. 
Therefore, a modest increase in spacing between adjacent fibers is 
usually sufficient to reduce the optical coupling to tolerable values. 
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Yet, in some cases, one needs to minimize the cross section of the cable 
and the spacing between adjacent fibers. Let us briefly discuss a few 
relevant applications. The need for minimizing the distance between 
single-mode cores in a fiber does not arise in communication systems 
presently envisioned for the following reasons: The fiber diameter is 
required to be large (e.g., larger than about 50 um) so that the fiber 
is able to sustain mechanical tensions. Thus, quite a few cores can be 
accommodated within the fiber diameter with sufficient spacing. 
Furthermore, the capacity of single-mode fibers is so large there is 
little incentive to introduce more than one core in the same cladding. 
The problem of coupling between single-mode fibers (or between fibers 
carrying few modes) does arise, however, when one tries to increase the 
image-transmission capacity of a fiber bundle up to the diffraction 
limit, each core carrying one bit of image information. Crosstalk 
(image blurring) is minimized if adjacent cores are made dissimilar. 
However, geometrical irregularities may restore a large coupling be- 
tween closely spaced cores. (This, incidentally, raises the possibility 
that measurement of the coupling between dissimilar, closely spaced 
cores gives useful information on the spectral density of the core 
irregularities.) The problem of coupling between single-mode dielectric 
waveguides also arises in integrated optics and in biology in the study 
of the optical behavior of the retina. The results that we present are 
general. They are therefore applicable, in principle, to multimode, as 
well as to single-mode, fibers. However, in practical multimode fibers, 
slow longitudinal variations of the core dimensions make the propaga- 
tion constants of the modes of one core sweep randomly through the 
propagation constants of the modes of the other core. Thus, an aver- 
aging takes place that cannot be ignored. The problem of coupling 
between highly multimoded cores will be only briefly discussed. 

The shielding method discussed in this paper consists of the intro- 
duction of a layer of metal, typically silver, between the adjacent 
optical waveguides. A reservation is in order: In some communication 
systems, metallic layers may be undesirable because they detract from 
the all-dielectric-cable properties. Shielding between adjacent fibers 
can be provided alternatively by low-refractive-index plastics such as 
Teflon® rep (n 1.32) that cause the optical field to decay faster 
than in the cladding material. The reduction in coupling, however, is 
much smaller than that provided by metals. Plastic materials can be 
made very lossy by impregnating them with dyes. High losses, how- 
ever, are much less effective than small refractive indices in reducing 
evanescent wave coupling. Therefore, we shall consider mainly 
metallic layers. The practicality of metallic shields remains an open 
question. 
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In the first part of this article series, a general and simple expression 
of the coupling between two lossy open waveguides was derived. Our 
formulation requires that only the normalized fields of the individual 
waveguides along a contour be known. In the present paper, we eval- 
uate in detail the crosstalk between two parallel slabs caused by the 
electromagnetic coupling and means of reducing it. The crosstalk 
between two optical slabs has been evaluated by Marcuse,? although, 
in Marcuse’s work, the slabs are assumed identical. In reality, un- 
avoidable fluctuations in the slab dimensions reduce the crosstalk, as 
we shall see, by more than 40 dB. Marcuse has also evaluated the reduc- 
tion of crosstalk provided by a layer of absorbing material located 
between the slabs. He found that the waveguide loss increases to in- 
tolerably high values before any significant reduction in coupling can 
be obtained. We find that, if the intermediate layer is metallic, the 
coupling can be drastically reduced without any significant increase 
of the waveguide loss. This discrepancy results from the fact that, for 
metallic layers, the permittivity is negative. For very dissimilar media, 
the first-order perturbation used by Marcuse is not applicable. In the 
present paper, we assume that the perturbation caused by the inter- 
mediate layer on the propagation is small, but we do not assume that 
the field in that intermediate layer is close to the field that would 
exist in the absence of the layer. 

In Section II, we evaluate the crosstalk between optical waveguides 
when the axial wave numbers (or propagation constants) of the iso- 
lated guides fluctuate along the system axis. In Section III, we eval- 
uate the spacing between slabs corresponding to a given crosstalk. In 
Section IV, the transmission is evaluated of a metallic layer under 
evanescent wave excitation and the crosstalk reduction. In Section V, 
we evaluate the loss that results from the introduction of a metallic 
layer near a slab waveguide. In Section VI, a simple approximate 
formula is given for the coupling between oversized round fibers. It 
is compared to exact results. Finally, brief comments are made in 
Section VI concerning the applicability of quasi-ray optics techniques 
in evaluating the coupling between irregular oversized fibers and the 
effect of bending. A few general results that do not seem available in 
convenient form in the literature are derived in the appendices. 


Il. FAST COUPLING 


Solution of the coupled-mode equations when the axial wave num- 
bers of the isolated guides are constant, or vary linearly with 2, is 
recalled in Appendix A. In the present section, only the results are 
given. 
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Let us first assume that the coupling c between the two guides and 
the axial wave numbers ky, ke, of the isolated guides is constant 
(independent of z). Let a power unity be fed into guide 1 at z = 0 and 
the other guide, guide 2, be unexcited. The power in guide 2 grows, at 
first, according to the law (see Appendix A) 


P2{z) = (cz)?. (1) 
This result is valid only as long as Az « 1, where we have defined 
A = {[ (ki — he)?/4] + 7}. (2) 


For example, a —20-dB crosstalk (P2 = 0.01) over a 1-km length 
of cable is obtained, according to (1), if c = 10-4 m~. Condition 
Az <1 is, for identical guides, z<«<10 km. However, if (ki — k2)/ 
(ki + ke) = 10-4, law (1) is applicable only if z<<1 mm, a dras- 
tically different condition. In Section III, the distance between the 
guides that corresponds to this particular coupling is evaluated. 

Now let ki — ke vary linearly with z. The coupling c remains a 
constant. We write 


kyi(z) = ko + a2, ko(z) = ko — az, (3) 


where ko and a denote constants. At large |z|, the coupling is insig- 
nificant because of the large value of ky — ky. The coupling becomes 
important only near the origin, 2 = 0, where near-synchronism is 
achieved. Let a power unity be fed into guide 1, at large negative z. 
The power transferred to guide 2 at large positive z is exactly (see 
Appendix A) 

P, = 1 — exp (—7c?/a). (4) 


We are interested in the case where the k’s are crossing very rapidly. 
Thus, let us assume that a is large and that, consequently, mc?/a is 
small. In that approximation, 


P, = e/a <1. (5) 


In most practical systems, k; — ke oscillates as a function of z. A sig- 
nificant amount of coupling between two guides takes place only near 
the crossing points. To develop an understanding of the effects of 
longitudinal variations of the difference of the axial wave numbers 
ki(z) and k2(z), we model the difference in wave numbers as a simple 
sinusoid, i.e., 

3 (ki — ke) = Sd sin (2), (6) 


where 6 denotes the peak deviation of (k; — kz)/2 and 27/Q the period 
of oscillation. It seems reasonable to assume that the phases of the 
signals picked up by fiber 2 at the successive crossing points are un- 


1434 THE BELL SYSTEM TECHNICAL JOURNAL, OCTOBER 1975 


correlated and that, consequently, the powers add up. This inco- 
herency is a consequence of the fluctuations of the phase of the optical 
field between successive crossing points. According to (6), the slope 
a introduced in (8) is 

a= 6Q. (7) 
The number of crossing points over a length z is 02/7. Thus, the power 
collected by guide 2 over length z is 


Py = (Q2/m) (re?) /(6Q) = 02/6. (8) 


Note that P, is independent of Q. P2 is proportional to c?, as was the 
case in the absence of fluctuations, but it varies linearly with z rather 
than being proportional to 2?. Let us compare P» in (8) and P» in (1). 
The ratio of these two collected powers is 


P, (uniform fibers) _ 
P, (nonuniform fibers) 


(9) 


It seems reasonable to assume that, over a length of 1 km (z = 10° 
um), the relative variations of the axial wave number are larger than 
10-4: 6/k > 10-*. For the single-mode slab considered in the next 
section, this number corresponds to a fluctuation of the slab thickness 
of 0.01 um. Because & is of the order of 27 pm, the reduction in cou- 
pling owing to the lack of identity between the two slabs is, in that case, 
of the order of 50 dB. The results obtained are therefore much too con- 
servative if we assume that the optical guides are identical in evaluat- 
ing the crosstalk. 


It. EVALUATION OF COUPLING BETWEEN TWO SLABS 


Let us consider two identical dielectric slabs having thickness 2d 
and material free wave number k. The free wave number in the medium 
between the slabs (cladding) is denoted k,, and the spacing between the 
slabs is denoted 2D. (See Fig. 1. The intermediate layer is to be ignored 
for the moment.) The expression for the coupling c between the fun- 
damental H waves is well known (see, for example, Ref. 1): 


c = kR exp (—2x«D), (10) 
where 
k = (k? — k)} _ (11a) 
R= (k.d) "(1 + (1/ed)J [1 — (x0? /F*) |] (11b) 
F2 = (k? — k2)d?. (11c) 


k, denotes the axial wave number of the isolated slabs (previously 
denoted k; and ke for the two waveguides). If we require that only one 
H mode propagate (for simplicity, we shall ignore the # waves), the 
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Fig. 1—Coupled dielectric slabs with thickness 2d and free wave number k. The 
cladding medium has free wave number k,. Crosstalk can be reduced by introducing 
a metallic layer with free wave number k,, (almost purely imaginary) and thickness 


me 


maximum value of F is 7/2. The theory of dielectric slabs shows that, 
for that value of F, xd = 1.28. Thus, the coupling is 


c = (0.24/kd?) exp (—2.56D/d), (12) 


where we have made the approximation k, & k in the first term. Thus, 
for a constant relative spacing D/d, the coupling between two single- 
mode slabs varies as the inverse of the square of their thickness. 


Let us evaluate c for the numerical values 
2d = 1.32 um, k = 29 X 1.45 um, ke = 29 X 1.4ym7!. (138) 
Thus, 


F = 7/2, k.d&kd = 8.88. (14) 
If we substitute these results in (11b) and (12), we obtain 
R = 0.021, cQG@n m7) = 4 X 104 exp (—3.88D), (15) 


where D is in pm. If the slabs are identical, —20-dB crosstalk in 1 km 
is obtained, as we have seen in Section II, when c = 10-4 m~. This 
corresponds, according to (15), to a spacing 


2D = 11 um. (16) 


If the slabs have some irregularities, with 6/k = 10-* (corresponding 
to a variation of slab thickness of 0.01 um), —20-dB crosstalk is ob- 
tained when c = 0.25 m7. This coupling corresponds to a smaller 
spacing: 2D = 6.2 um. If the slab thickness is chosen equal to 10.5 
um, keeping F = 1/2 (An/n = 5 X 107), the spacing required for 
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identical slabs and —20-dB crosstalk over a 1-km length is 2D = 66 
pm, a rather large spacing. 

If the two slabs are uniform but are made deliberately dissimilar, 
a lower crosstalk is obtained. The relative difference 5/k in axial wave 
numbers is approximately 0.5 (Ad/d)/(kd)?, where Ad/d is the relative 
difference in thickness of the two slabs (F = 7/2). For example, if 
one slab has a thickness 2d equal to 1.32 wm and the other has a thick- 
ness equal to 1.2 um, the relative difference in k. is: 6/k = 0.65 X 107%. 
The maximum relative power that can be transferred from one slab 
to the other is, according to eq. (39), equal to (c/A)?, where A 6. 
Thus, a —20-dB crosstalk corresponds, for the above value of 6, to a 
coupling c = 6/10 = 580 m—!. The slab spacing 2D corresponding to 
that coupling is given by (15). We obtain 2D = 2.2 um. More generally, 
we find that D & 1.5d for any d, if F is kept equal to 7/2 and Ad/d 
= 0.1. Thus, a considerable reduction in spacing is tolerable, in prin- 
ciple, if the slabs are made dissimilar. However, fast fluctuations 
along the z-axis of the slab dimensions with a period of the order of 
az/5 = 100 um would reestablish synchronism between the two slabs. 
Fluctuations that are too small in amplitude to deteriorate the prop- 
agation under normal conditions (e.g., no significant coupling to the 
radiation modes) may nevertheless introduce a large crosstalk when 
the slabs are very close to each other. Thus, the result obtained above, 
that the spacing between slabs can be reduced to 1.5 X (2d) if the 
fibers are made dissimilar, may not hold true in practical conditions. 


IV. TRANSMISSION THROUGH A METALLIC LAYER UNDER 
EVANESCENT WAVE EXCITATION 

The results in Section II show that the crosstalk power P2 is pro- 
portional to the square of the coupling c. We have shown in Ref. 1 
that, for identical slabs and a symmetrical configuration, the coupling 
c is proportional to the square of the normalized field halfway between 
the two slabs. Thus, the crosstalk is proportional to the fourth power 
of the normalized field halfway between the two slabs. If we introduce 
a metallic layer of thickness 2d, symmetrically centered between the 
two slabs as shown in Fig. 1, the crosstalk is reduced in proportion to 
the fourth power of the field in the middle of the metallic layer. This 
field reduction, denoted ¢ (for transmission), is evaluated in the present 
section. 

Let us consider an evanescent wave with axial wave number k, > k,, 
where k, denotes the free wave number in the medium. This wave 
decays in the x direction according to 


E(x) = Ey exp (—x«xz), (17) 
KR (k2 — k2)3. (18) 
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Let us now introduce a metallic layer with complex wave number 
km = km, + thm: and thickness 2d,. The ratio ¢ of the field in the 
middle of the layer to the field at the same point in the absence of the 
layer is derived in Appendix B. Provided the layer is sufficiently thick 
or, more precisely, that 


Real (kmdm) > 1, (19) 
where 
Km = (kz — ki,)}, (20) 
we have 
t = [Axkm/(k + km)? ] exp [(k — km)dm ]. (21) 


At a free-space wavelength \» = 1 um, ko = 27 upm™, the wave number 
of silver is almost purely imaginary,’ 


k2, = (Kime + tkmi)? = (0.2k0 + 25ko)? = —985 + 797 (in um-?), (22) 
and, for a typical glass, assumed lossless (n, = 1.4), 
| k2 = 22 = (1.4ko)? = 77.4 um. (23) 


With the value of x? = k2 — kf = 3.76 wm in (14), and k?,, k2 in (22) 
and (23), we obtain x, = 32 — 1.32, and, from (21), a power trans- 
_ mission 


I 


T = tt* = 0.062 exp (—60d,n), (24) 
where d» is in wm, provided 
dm >> 0.03 ym. (25) 


Because the crosstalk power P2 is proportional to the square of the 
power transmission 7’, the introduction of a layer of silver of thickness 
2d, between the two slabs reduces the crosstalk in dB by 


20 logio (1) = 520d,n, (26) 


where d» is in um. For example, if the layer thickness is 2d, = 0.5 um, 
the crosstalk is reduced by 130 dB. This reduction is independent of 
the initial value of the crosstalk, within the approximations made. 
Thus, a 0.5-um-thick layer of silver is sufficient to ensure a complete 
isolation of adjacent fibers, at a wavelength \» = 1 um. 

Surface polaritons can be guided near the dielectric (k2 > 0) and 
metallic (k?, < 0) interface. However, the losses of such modes are 
extremely high over a distance of 1 km. The cladding modes are also 
strongly attenuated, and it seems that they can be safely ignored. For 
comparison, let us consider, in place of the metallic layer, a low-index 
plastic material of the Teflon type, with a refractive index n = 1.32. 
We now have k?, = 69 um~? and km = 3.47 um. We obtain a cross- 
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talk reduction equal to 26dm in dB, where dm is in ym. Thus, a 50-dB 
reduction in crosstalk requires a 4-um-thick layer of low-index plastic 
material. 


V. LOSS INTRODUCED BY A METALLIC LAYER 


We are now concerned with the fact that, because the refractive 
index of a metal is not purely imaginary, the presence of the metallic 
layer may increase significantly the loss of the modes guided by the 
fibers. This loss depends critically on the distance between the metallic 
layer and the fibers and, therefore, on the distance between the two 
fibers. The loss suffered by the fiber is influenced by the complex re- 
flection of the metallic layer for evanescent waves. This reflection, 
strictly speaking, depends on the thickness of the metallic layer. Exact 
expressions are given in Appendix B. However, in all our numerical 
examples, the thickness of the metallic layer is so large that it can be 
assumed infinite. In that case, the reflection r reduces to 


r= (kK — km)/(k + km); (27) 


where «x and km are defined in (18) and (20), respectively. Because the 
imaginary part km: of km 1s much smaller than the real part km, the 
imaginary part 7; of r is approximately 


15 FS kK mi/ Keng. (28) 


If we use for kz, ks, and k» the numerical values in (14), (23), and (22), 
respectively, we find r; = 0.005. 

To obtain the loss suffered by the slab, we use the perturbation 
formula derived in Appendix C. The variation of k, is assumed to be 
small. The variation of the field near the perturbing object, however, 
is not assumed small. In the present case, kz is real before perturbation. 
The introduction of the metallic layer causes kz to acquire a small 
imaginary part, k.;. The imaginary part k.: of k, is the fiber loss, in 
neper/unit length. There is also a small variation of the real part of 
k,. This variation, however, is of no interest to us. We have (see 
Appendix C) ; 

kaw = rixR exp (—2kD,,), (29) 


where F is the slab parameter defined in (12a) and D,, the distance 
between the slab and the metallic layer. The imaginary part r; of the 
metallic layer reflectivity is given in (28). 

For the numerical values used earlier in (14) and (15), we obtain 
from (29) 


Lasjkm = 8.7 X 10%. = 2.6 X 108 X exp (—3.88Dm), (30) 
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SILVER LAYER THICKNESS IN jum 


Fig. 2—Reduction in crosstalk and increase in fiber loss resulting from the intro- 
duction of a silver layer of thickness 2d,, (free-space wavelength = 1 um). The dielec- 
tric slabs have a normalized frequency F= (k? — k?)!d = w/2. Their spacing is kept 
equal to 11 wm. The loss varies with d,, only because of the change in the slab-layer 
spacing. In the absence of metallic layer, crosstalk is —20 dB/km. 


where D,, is in ym. For Dx, = D — dm = 5.25 um, the loss introduced 
by the metallic layer, given in (80), is only 


& = 0.017 dB/km. (31) 


This loss is quite negligible compared with the other losses (absorption 
because of impurity or scattering losses) suffered by the wave. How- 
ever, because £ depends critically on D,,, this loss may not be negligible 
in all practical cases. The reduction of the crosstalk and the increase 
of loss caused by a silver layer of thickness 2d,, are shown in Fig. 2 for 
the dielectric slabs considered earlier, as functions of 2d,,. Note that, 
if we assume for simplicity that the thickness of the metallic layer is 
negligible compared with the slab spacing (2d, << 2D), the (dimen- 
sionless) ratio of k,; (loss) and c is, approximately, 


kii/¢ = 2(kmr/kins) Kd. (32) 


Thus, the best metal, from the point of view of propagation, is the 
one whose kn-/k74 is the smallest. 


VI. ROUND FIBERS 


The general coupling formula in Ref. 1 is applicable, in principle, 
to round fibers. Round fibers are more often encountered in practice 
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than are slabs. The geometry is shown in Fig. 3. The fibers are assumed 
identical, with radius a and spacing 2D. The results are given only for 
the scalar fundamental field y 2 HE of oversized fibers [F = 
(k? — k2)?a >> 1]. In that approximation, the normalized field is 
easily found to be (see Part II of Ref. 1) 


¥(y) = uo(mkiaF)~ exp (—Fy?/2a’), (33) 


where uo 2.4 --- is the first zero of the Bessel function of order zero. 
The y axis is tangent to the rod considered, as shown in Fig. 3. The 
Fourier transform of (y) is 


I 


A 1 +0 é 
Dk) = (2m) a ¥(y) exp (—thy)dy 
= mtuok?F-3 exp (—k2a?/2P). (34) 


Because the spectral component ¥(k,) varies approximately as 
exp (—sx) as a function of 2, where s = (k2 —k3)? 2 F/a, the 












EXACT 
mmm APPROXIMATE 





D/a 


Fig. 3—Variation of the coupling between two dielectric rods of radii a as a func- 
tion of their spacing (2D). The dimensions and free wave numbers are shown. The 
parameter K is defined as ca(l — k2/k?)—, and c is the coupling. The plain lines are 
from Snyder exact theory,’ and the dashed lines from the theory in Ref. 1, applied 
to large normalized frequencies F. 


TRANSVERSE COUPLING IN FIBER OPTICS IV 1441 


coupling is 
c= i. 8(ky)bo(ky)Wi(ky)dky 


= (ud/m?)kF-ta~ exp (—2FD/a). (35a) 
In place of c, we can use a normalized coupling K defined by 
K = call — (k2/k?)]-}. (35b) 


In the general expression for c in (34), %2 and 1 represent the spectral 
components of the field of the two fibers along the y-axis at x = 0. 
The normalized coupling K is plotted in Fig. 3 (dashed lines) as a 
function of the ratio D/a of the fiber spacing (2D) to fiber diameter 
(2a). In that figure, the parameter is the normalized frequency F’. For 
comparison, an exact result obtained by Snyder‘ is shown as a plain 
line. The agreement is very good for F > 4. 

The advantage of the method used in this section is that it is ap- 
plicable when the two fibers are separated by a metallic layer. In that 
case, one need only introduce inside the integral sign in the first ex- 
pression in (34) a term 7'(k,), where T denotes the power transmission 
of the metallic layer, defined in (24). T now depends slightly on k, 
because, in the expressions given earlier for 7’, the axial wave number 
k, should be replaced by (k? + k?)*. The effect of the dependence of T 
on k, is small, however, and the value obtained earlier for 7 for slabs 
is approximately applicable to round fibers as well. 


Vil. MULTIMODED IRREGULAR FIBERS 


We shall make only qualitative comments. In the preceding calcula- 
tions, we have considered the coupling between one mode of one core 
and one mode of another adjacent core. If the cores can carry many 
modes and have dimensions that fluctuate as a function of z, with such 
an amplitude that the variations in axial wave numbers exceed the 
spacing (in axial wave numbers) between adjacent modes, some averag- 
ing takes place. The situation becomes comparable, at least over some 
distance, to that of a slab radiating power into a semi-infinite dielectric, 
a situation discussed in detail in Part II of this series of papers.! 

Let us picture the field in slab 1 (excited at z = 0) as made up of 
two plane waves. The plane wave moving toward slab 2 tunnels into 
slab 2. Because of the fluctuations in axial wave numbers, the power 
transferred from slab 1 to slab 2 is essentially the power carried by 
that tunnelling wave; we can ignore the fact that this wave, after 
tunnelling, is reflected back and forth inside slab 2 and may tunnel 
back to slab 1. The power transferred from slab 1 to slab 2, then, is 
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proportional to z, for small z, rather than to the square of z, as is the 
case in the absence of irregularities. This picture is consistent with 
that used by Cherin,® who adds the powers transmitted by tunnelling 
rays. Let us emphasize that the validity of this quasi-ray optics ap- 
proach rests on the presence of large slow fluctuations of the core 
dimensions. A simple calculation shows that the relative fluctuations 
of the slab thickness must exceed the reciprocal of the mode number. 
This condition is never met for the low-order modes, but it may be 
met by the higher-order modes. Thus, the situation is rather compli- 
cated and requires a deeper analysis. This quasi-ray technique should 
not be confused with that of Kapany and Burke,*® where the slabs 
are assumed identical and the fields of the tunnelling rays, rather 
than their powers, are added. In the preceding discussion, we have 
assumed that the fiber cable is essentially straight. The coupling 
increases significantly if the cable is bent.? This effect makes it even 
more important to provide shields between adjacent fibers. 


Vit. CONCLUSION 


We have shown that a drastic reduction of crosstalk between parallel 
dielectric slabs can be obtained by introducing a layer of silver (thick- 
ness & 0.5 um) between adjacent slabs. The reduction, in decibels, is 
proportional to the imaginary part of the refractive index of the metallic 
layer and to the layer thickness. In many cases of practical importance, 
the loss introduced by this metallic layer is negligible. We have also 
shown that, because of unavoidable irregularities in the fiber dimen- 
sions, the crosstalk is at least 40 dB below that expected for identical 
fibers. 
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APPENDIX A 
Fast and Adiabatic Coupling 
Let y denote the field of a guide, such that yy* is the power. When 


two guides are weakly coupled, their respective fields ~1, wo approxi- 
mately satisfy the well-known equations® 


—idp,/dz = ki(z)vn + Che 
—tdib,/dz = ko(z)We + cy. 
For simplicity, we assume that the axial wave numbers ki, ke of the 


isolated guides are real and that the coupling c is a real constant. The 
solution when ki, k2 are constant is well known. For the convenience 


(36) 
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of the reader, this solution is derived below. The general solution of 
(36) is a superposition of normal modes . 


¥i(z2) = Wi exp (dktz) + Yr exp (ck-z) (37) 


v2(z) = pe exp (ktz) + wo exp (zk-2), 
where 
k+ = (ki — ko)/2 4 A (38a) 
A = {[(ki — ke)?/4] + c?}}. (88b) 


If the initial conditions are ¥1(0) = 1, ¥2(0) = 0, that is, if only 
guide 1 is excited at the origin (¢ = 0), the field in the unexcited guide, 
2, is 


Wo(z) = (ic/A) exp [2t(ki + ke)z/2] sin (Az). (389) 
Thus, for small z, the power in guide 2 increases as 
P3(z) = (cz)?, hed. (40) 


This result is independent of ki — ks. See Fig. 4. 


COUPLED IN POWER 





cz 


Fig. 4—Variation of the optical power picked up by fiber 2, where only fiber 1 is 
excited at z = 0, as a function of the normalized axial distance. The axial wave 
numbers of the isolated fibers are assumed to be constant but different [parameter 
(k, — k,)/2c]. Note that the behavior for small cz is independent of ki — ko. 
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(b) 


Fig. 5—(a) Linear variation of the axial wave number of the isolated waveguides 
as a function of the axial coordinate z. The hyperbolas represent the normal mode 
wave numbers. (b) Adiabatic coupling in fiber optics. All the power from one fiber 
is transferred to the other fiber if the k’s vary sufficiently slowly. This principle is 
applicable to multimode fibers. 


Let now the axial-wave numbers k,, kz of the isolated guides vary 
linearly with z 
ki(z) = ko + a2, ko(z) = ko — az, (41) 


where ky and a denote constants. Synchronism takes place only near 
the origin, z = 0. Let us set 


¥1,2(2) = A1,o(z) exp (tkoz) (42) 


in (86). After differentiation and substitution, we obtain an equation 
for A, 
(d?A1/d2?) + (a?22 + c — ia)A; = 0. (43a) 


A similar equation holds for Az that we need not write down. Equation 
(43a) is the equation for parabolic cylinder functions. The asymptotic 
form of the solution, valid for —7/2 S arg(z) S a is, for a power 
unity at 2 = —o (see Ref. 9), 


Ai(z) = exp [7(a/2)2? + 7(c?/2a) log (—z) J, 2<c/a (43b) 
A1(@) = exp [t(@/2)z? + 7(c?/2a) log (2) — re?/2a],  2>>Cc/a, (48¢) 


as we easily verify by substituting (48b) in (43c) and neglecting terms 
of order 2. To go from (48b) to (48c), note that log (—2) 
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= ix + log (2). Note also that a change in the unit with which z is 
measured affects only the amplitude of Ai, which is arbitrary. 

The power in guide 2 after the interaction has taken place, that is, 
for large positive z, is, according to (48c), 


P2=1- A,Aj =1- exp (—1c?/a). (44) 


Let us first assume that ac?/a is very small compared with unity, 
that is, the k’s are crossing very rapidly. In that case, guide 1 transfers 
only a small amount of power to guide 2, equal to mc?/a. This is the 
result used in the text. 

When zc?/a is very large compared with unity, that is, when the 
variation of ki — kz is very slow, almost all the power from guide 1 
is coupled to guide 2. This is the principle of the Cook adiabatic 
coupler.!° This mechanism is applicable also to multimode dielectric 
waveguides. It may be used to couple two optical fibers because the 
dimensions are not critical. Only slowness is required.!! (See Fig. 5.) 


APPENDIX B 


Transmission and Reflection at a Metallic Layer Under 
Evanescent Wave Excitation 


Let the metallic layer have a complex free wave number kn = kr 
+ tkm: and a thickness d». The surrounding medium is assumed to 


have a real free wave number k,. The field has the general form (see 
Fig. 6) 


E,lexp (—«x) + r exp (xx) ], x0 
E(2) = {e exp (—Kkmx) + Et exp (kn) O<x<ezd, (45) 
Et exp (—xz) Napa fee 
where 
x == (kz — k5)} (46) 
is real, and 
Km = (k2 — k3,)}. (47) 





k 


Ss 


Fig. 6—Transmission of a metallic layer with thickness d,, and free wave number 
km under evanescent wave excitation (axial wave number k, > k.). At large negative 
zx, the field is assumed unperturbed by the layer. 
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The axial wave number k, is assumed to be real and larger than k,. 
By specifying that # and dE/dx are continuous at the boundaries, 
x = 0, x = dm, we obtain the reflection r and the transmission t: 


L(k/km) — (km/«) L2 coth (mdm) + (Km/«) + (k/Km) (48) 
exp (kdm) {cosh (Kmdm) + $[(km/k) + (k/km) | sinh (kmdm)}—1. (49) 


We shall now assume that the metallic layer is thick in the sense 
that Real (kmdm) >> 1. These conditions are well satisfied for the 
metallic layers that we consider in the main text. In that case, (48) 
and (49) reduce to 


I 


r 


t 


I 


(k — km)/(k + km) (50) 
[4xkm/ (kK + km)? ] exp [(k — km)dm_], (51) 
respectively. Equations (50) and (51) are the results used in the text. 


r 


t 


APPENDIX C 
Loss Introduced by a Metallic Layer 


Let us consider a uniform reciprocal waveguide and let a uniform 
rod be introduced that perturbs the propagation of the waveguide 
(Fig. 7a). We assume that the perturbing rod does not support trapped 
modes or, if it does, that the axial wave numbers of these trapped 
modes are sufficiently far away from that, k.o, of the waveguide. No 
resonant coupling is assumed to take place. 

We shall first recall a very general result. Let E+, Ht and E,, H, 
denote two time-harmonic fields at the same frequency in the same 
medium. If we assume that the medium is reciprocal (that is, that the 
tensor permittivity is symmetrical), it readily follows from the Maxwell 
equations that the divergence of the vector 


J=E*XH,+H* XE, (52) 


is equal to zero. Thus, the flux of J through any closed surface is equal 
to zero. In what follows, an exp (—7wt) term is omitted. 

Now let E+, H+ be the field propagating in the —z direction along 
an open waveguide. The dependence of E+ and H+ on z is denoted: 
exp (—7k.0z). Let Ep, H, be the field propagating in the +2 direction 
in the presence of the perturbing rod with an exp (ik.z) dependence 
on z. The closed surface S is taken as the surface shown in Fig. 7a 
bounded by the planes z = 0 and z = dz, the volume of the perturbing 
rod being excluded. For that choice, the medium enclosed by S is the 
same for both fields. We can therefore use the result stated earlier that 
the flux of J through S is zero. Let us consider the various contributions 
to that flux. The flux of? J through the plane z = dz differs from the 
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PERTURBING ROD 
( 





(b) 


Fig. 7—(a) Schematic for the derivation of the general perturbation formula. The 
dielectric waveguide is perturbed by a small lossy rod. The closed surface S extends 
to infinity where the fields considered are assumed to vanish. (b) Application to the 
perturbation of H waves guided by a dielectric slab (k) by a lossy slab (Kn). 


flux of J through the plane z = 0 only by a factor —exp [7(kz — kzo)dz ]. 
The difference between these fluxes is, therefore, 7(k. — kzo)dz for 
small dz. Because we are considering only trapped modes, the flux at 
infinity is zero. The flux through the surface surrounding the perturb- 
ing rod is dz times the line integral of J-dC, with dC a vector perpen- 
dicular to the contour surrounding the rod, pointing inward, whose 
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length is the elementary arc length. Thus, we have exactly 


Ake I J-c uh ic J-dS, (53) 


Ak, = kz ro kz. (54) 


So denotes the transverse plane, z = 0 minus the area enclosed by C, 
and dS denotes a vector directed along the z axis whose length is the 
elementary area. The derivation given above is almost identical to 
that in Ref. 1 for coupled waveguides. We now assume that the per- 
turbation is small. Thus, we can replace Ey, H, by the unperturbed 
field E, H propagating in the +2 direction in the integral over So in 
(53). This is not permissible, however, for the integral over C, in 
general. 

Let (52) be specialized to the H waves guided by a dielectric slab 
shown in Fig. 7b. In that case, E has only one component: L, = E(z), 
H,= (1/iwu.)dH/dxz, and H, = —(k.z/wuo)E. Taking into account 
E; = E, and H+ = H, (see Ref. 1), we obtain 


Ak, = [(E0E,/dx) — (E,0E/dz)] ye (2k. iE = Bia) , (55) 


where 


where we have assumed that EF, differs significantly from # only near 
the perturbing slab. The unperturbed field is, for —Dm <x < 0, 


E = exp (—x«2), (56) 
and the perturbed field is that given in (45) 
E, = exp (—xx) + r exp (x2), (57) 


where 
x = (k2 — k2)}, 


The amplitudes in (56) and (57) are so chosen that EL, & E for large 
negative x, e.g.,v = —Dm. 
We first evaluate 


(EOE ,/dx) — (E,0E/dx) = 2rk, © (58) 


where we have used (56) and (57). Note that the result (58) is inde- 
pendent of x (for —D, <x <0). Substituting (58) in (55), the 
imaginary part of k, is found . 


ku = r«R exp (—2kD m), (59) 


where r; denotes the imaginary part of r, evaluated in Appendix B. 
We have introduced in (59) the field strength parameter 


t= (x. i. A Bide) (60) 
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In the above definition of R, the field is assumed to be unity at the 
guide-cladding boundary. For a dielectric slab, the value of FR is given 
in (12). Equation (59) is the result used in the main text. 
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Faster-Than-Nyquist Signaling 


By J. E. MAZO 
(Manuscript received March 27, 1975) 


The degradation suffered when pulses satisfying the Nyquist criterion 
are used to transmit binary data in noise at supraconventional rates 1s 
studied. Optimum processing of the received waveforms 1s assumed, and 
attention is focused on the minimum distance between signal points as a 
performance criterion. An upper bound on this distance is given as a 
function of signaling speed. In particular, the pulse energy seems to be 
the minimum distance up to rates of transmission 25 percent faster than 
the Nyquist rate, but not beyond. 

Some mathematical aspects related to the above problem are also con- 
sidered. In particular, the minimum distance is rigorously shown to be 
nonzero for all transmission rates. This is tantamount to showing that, 
an the singular case of linear prediction, perfect prediction cannot be 
approached with bounded prediction coefficients. 


I. INTRODUCTION 
The use of Nyquist pulses 


to send binary (or multilevel) data without intersymbol interference 
over a channel of bandwidth W = (1/27T)Hz is classic. If we assume 
that one receives the pulse train 


Ne 
u(t) = >) angié— nT), a, = +1, independently, (1) 
n=N1 


in additive white gaussian noise of two-sided spectral density No/2, 
then the optimum detector has a bit-error rate P, given by 


Pix o( 5) 2) 
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where 
Q(x) = = a eV Rdy = 4 erfe = (3) 
NQn Ja 2 v2” 


erfe (-) denoting the co-error function, and F# being the energy in the 
pulse g(t). In our case, H = T. Asymptotically, for large signal-to-noise 
ratios, (2) becomes 


Pwd Ntexp (- =): (4) 

We now address the following question : Suppose that in transmitting 

(1) we obtain a performance from (2) that is more than satisfactory. 

Thus, we may have a P, of 10-° or 10-7 when 10-° would be adequate. 

To what extent can we trade this ‘“‘excess performance” for speed by 

replacing T by JT’ < T in (1), while keeping transmitted power 
constant? In other words, we still use pulses 


g() = Be (5) 


but send them at intervals T’ < T. We call this faster-than-Nyquist 
transmission and shall characterize T’ by writing T’ = pT,0 < p <1. 
A particular motivation for this problem is to mathematically model, 
in a simple way, what would happen if voice-band telephone channels 
are “pushed” to their limits with more rapid transmission of pulses 
than has been conventional. 

While simple detectors that match filter and sample can still be 
used for faster-than-Nyquist transmission, their performance is 
suboptimum.! We are concerned here with optimum detectors. Since 
exact analysis of nonlinear detectors is not presently feasible, we 
choose to give our detectors the benefit of the doubt and work rather 
with lower bounds to P,. Nevertheless, interesting results can be 
obtained regarding the trade-off considered here. To see why degrada- 
tion in error rate is inevitable, note that (2) is the well-known matched 
filter bound for antipodal pulses, each of energy ZH, which must bound 
performance for bit detection with a sequence of (perhaps interfering) 
pulses. On the other hand, as J’ decreases, pulses are sent faster 
and the energy E in each pulse must be decreased in direct proportion 
so that the power H/T’ is kept constant. This is an immediate un- 
avoidable element in performance degradation, and may be regarded 
as a “fair” trade-off. Another cause of degradation is the degree to 
which the optimum detector can cope with the interference among 
pulses, i.e., the fact that the performance will drop below that of (2). 
Here, bounds other than (2) are useful, and in fact are the first item 
taken up in the next section. 
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Il. DISCUSSION OF LOWER BOUND FOR ERROR RATE 


Assuming (1) is received in white noise and an optimum detector is 
used for detecting the kth bit, a lower bound on the chance of making 
an error on this kth bit will now be derived. Since the data a, are in- 
dependent, this bound also serves for any sequence (1) starting at 

= N; S Ni, and ending at n = Nj = N>2. We begin with the fact 
that, for a binary hypothesis problem with equal a priort probabilities 
and having p,(x) or p_(x) as the two probability densities of the 
received signal « under the two respective hypotheses, one way? to 
write the probability of error is 


P.= 5 f min[p.(e), p-@) ax, (6) 


If we let u#(¢) be a particular one of the equiprobable 2” signals in 
(1), N = N»2 — Ny, which have +1 in the kth position, then formally 


pola) = ay D Peo) @) 


where p(x) is the density of the observations conditioned on the 
entire sequence. Thus, 


Py= san | min (> D4 (2); z oe 
2 5arD 3 _ if min Cpt. (x), p2 (x) ]dx. (8) 


In writing (8), we have made use of the fact that the minimum of two 
sums with an equal number of terms is at least as large as the sum of 
the minimum of the two 7th terms of each series. Of course, each series 
can be arranged in any permuted order before the pair-wise minimum 
is taken and, thus, the pairings 7 with j (2) are indicated in (8) to allow 
for this permutation. Now 


5 f min [p's @), p @) Ide (9) 
is the probability of error with two fixed signals and has the well-known 
evaluation 

dla, 7 (7) ] ) 
ee |, 10 
o( aR oe 
where 
@(i,j) = [- [uh (@) — wl Pat (11) 


is the ‘distance’ between two sequences (1) which differ in the kth 
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position. Equation (8) then reads 


1S o(4i@] 
Pw 20") ae 


for any set of pairings [7, 7(z) ]. The bound (12) is intimately related 
to Forney’s lower bound,’ although our derivation is quite different. 
Forney’s bound in the present situation reads 


> Genin ) 
P.2 paO( 7%), (13) 
where dmin 18 the minimum distance between signals (1) which differ 
in the kth position, and p, is the probability that a sequence chosen 
at random has a sequence with opposite polarity in the kth position 
at distance dmin. Equation (12) can be made to yield something like 
(13). Thus, in (12) discard all terms except for those pairings [7, 7 (2) ] 
such that d[7, 7(z) ] = do. Then (12) implies 





no. of pairings do 
P, = BOOTED a( 7): | ve) 


The coefficient in front of the Q function corresponds to the proba- 
bility coefficient in (18). Choosing do = dmin yields (13), but when 
we will not be able to find dmin, eq. (14) will serve our purpose. 


Il. ESTIMATING THE MINIMUM DISTANCE 


Clearly, in (14) we should like to find the smallest dy to maximize 
the lower bound, provided the coefficient is not too small. In our 


problem, d2;, is given by 


N 12 
1— > ae! da, (15) 
1=1 








fun ing St f™ 

dE oN. fie LG 2mp a 
where we have normalized by dividing by the pulse energy H. The 
expression (15) comes from taking the Fourier transform of (11) and 
manipulating the resulting expression slightly. We note particularly 
that in (15) only positive values of / need be considered, since 








M 2 M+K (2 
eis(a - > ae) -_ r — > dei 
I=—K 1=1 
#0 
if a_x ~ 0. We have set b; = —a_xa;,_x if 1 # K and bh = —a_xif 


(=k, 

We cannot claim to have found the minimum value of (15). How- 
ever, a simple numerical effort has yielded the results for dj/4# shown 
in Fig. 1, where do refers to the smallest distance we have found. We 
note in particular that do is the pulse energy for p decreasing from 1 
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1.0 0.8 0.6 0.4 0.2 
p 


Fig. 1—The smallest distances between signal sequences that we have found are 
shown here for different values of signaling rate. Labeling a point by K indicates 
that the polynomial is 


K 
pe) td Cale 


to 0.8, or, in other words, for rates exceeding the Nyquist rate by 
25 percent [percentage of excess = 100(1/p — 1)]. Thus, d2in/4 
cannot be the pulse energy for p < 0.8 for this problem. By the time 
p has decreased to 0.5, d§/4EH has dropped to 0.465. (G. J. Foschini 
has informed the author that the use of the polynomial p(z) = 1 — z 
+ 23 — 2t+ 26 — 27, 2 = exp (20), results in the value 0.410 for 
dj/4E at p = 0.5.) Except for some points in the neighborhood of 
p = 0.4, the values for dj have been obtained by considering numeri- 
cally the best value of K which minimizes, for not too large K, 


1 pr |. K _ (2 
= / 1+ 5 (-1)e*| ao. (16) 
2m Pp: J =p% l=1 


These points are labeled with the appropriate value of K in Fig. 1. 
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Somewhat surprisingly, the larger values of K are responsible for 
decreasing do initially (K =7 at p = 0.8), and then K gradually 
becomes smaller (K = 2 at p = 0.5). The value obtained with K = 1 
always was suboptimum, as was the limiting value of (16) when 
K —~, which is easily shown to be 
= tan ©. (17) 
Tp 2 
Why were the sequences given in (16) deemed to be of interest in 
the first place? The most interesting reason stems from the following 
argument. If one considers the Fourier transform of a doubly infinite 
pulse sequence like (1) when pulses are being sent faster than Nyquist 
and when the special case of the alternating sequence a, = (—1)" is 
being sent, one finds that the Fourier transform consists of delta 
functions spaced at all odd multiples of 7/7’, that is, the Fourier 
transform is out-of-band, which suggests zero received energy. Ac- 
tually, the doubly infinite model and its 6-function Fourier transforms 
are idealizations representing limiting behavior for signals consisting 
of pulses extending from (—N,N) and N becoming large. We are 
really concerned with limiting behavior of the energy contained in 
the frequency interval (—72/T7,7/T), with T > T’, and evidently for 
the present case, if Sy(w) is the Fourier transform of the truncated 
pulse sequence, 


1 


oper 
lim — J Site) (tae 
—n/T 2a 


No 27 


r/T 
/ lim Sw(w)|?dw = 0. (18) 
—7/T 


In spite of the above subtlety, however, sequences which are alter- 
nating at least over part of their range are interesting and one might 
expect difficulty distinguishing between one such sequence and its 
negative. 

In addition to the normalized distances given in Fig. 1, Fig. 2 plots 
the numerical values of lower bounds computed from expression (14), 
as well as the matched filter bound. These curves all assume constant 
power. Curves with initial (9 = 1) error rates with 10-5 and 10-7 are 
chosen as examples in Fig. 2. In both cases, an order of magnitude of 
degradation in error rate is seen for a 25-percent increase in bit rate 
(o = 0.8) using only the matched filter bound. Decreasing p further 
on the 10-’ curve illustrates further degradations using (14) with an 
appropriate value of K. These bounds do not show a departure from 
the matched filter bound for as small a value of p as Fig. 1 would 
suggest, because the coefficient 1/2* to be used in (14) swamps the 
effect of the decreasing ‘‘minimum”’ distance. For the 10-° curve, 
this effect extends to even smaller p and no lower bound other than 
the matched filter one is shown for that case. 
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5 
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LOWER BOUND FOR P, 


= 
Oo 
t 

or 


1076 





1.0 0.9 0.8 0.7 0.6 0.5 0.4 
p 


Fig. 2—Lower bounds on error rate vs signaling speed for two initial (p = 1) 
cases. The solid curves are both matched filter bounds. The dashed curve is based 
on minimum-distance considerations and applies to the 10-7 case. All curves are 
drawn for constant power. 


IV. TWO MATHEMATICAL QUESTIONS 


As we have already emphasized, the infimum of the right member 
of (15) over all the indicated trigonometric polynomials with +1, 0 
coefficients is not displayed in Fig. 1. Figure 1 simply shows the 
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smallest values we have found. Next, we want rigorously to establish 
here that d2.,, ~ 0 if p ~ 0. Note that this would not be the case if 
the coefficients a; in (15) were allowed to be any real numbers. In 
fact, for any nonnegative function f(@) with In f(@) € Ii(—7, 7), 
we have the Szego theorem‘ which states 


it ae F(6) “do = exp 5- [ In f(9)d6. (19) 


Expressions such as (19) occur, in particular, in linear prediction 
theory. 

In our case, f(@) = 0 if |6| > px and Inf(6) is not Ly, but the 
appropriate limit of (19) indicates zero to be the infimum, which is 
the correct answer. Thus, there is some cause to wonder if d2j, as 
defined in (15) is zero as well. We shall in fact show it is slightly more. 








Theorem 1: Let 8 be any positive (finite) real number and require |ai| S 8B, 
l= 1,2, ---. Then 


: 1 pu 
inf — J 
N; {az} 2m J_ox 


Proof: We first note that if there exists a sequence {pn(@)},=1 of 
trigonometric polynomials of the form 


N 2 
1— ¥ ae’! da> 0, p #0. (20) 
T 








pr(6) = Lane, |u| $6 < « (21) 
such that 
= eo |i — p,(6)|2d0 0, (22) 
thaw for enge Olen 
| : G(6)p,(6)d0 > i . G(0)d6. (23) 


This is simply a statement of the fact that if p,(@) converges strongly 
to unity, it also converges weakly to unity. Now it is easy to see from 
(23) and the form of p,(@) that 


B »B | “ab Go) > | C0) (24) 
Or, in other words, if 
il " G(@)a6) 
Bx sup — ; (25) 








s ecto) 


—pr 


TIn addition to G(@) € L2(—pz, pr) it will sometimes be convenient to regard 
G(@) € L2(—7x, +) but having support confined to (—pz, pz). 
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then (22) cannot be true. In particular, if (25) holds with @ = 1, then 
d24n 18 strictly positive. Regarding G(@) € Le(—7z, 7) but supported 
on [—p7, pz], and calling 


g) = 5 [" eG @as, (26) 


gn = g(n), 
the right member of (25) contains the quantity 


|9o| 2 (27) 
X ga 


Clearly, we have a question concerning the sample values g, at the 
nonnegative integers of a function whose bandwidth is strictly less 
than 7. Normalizing (27) with go = 1, (25) prompts the question: 
How small can °° |gn| be? If it can be zero, then (25) would be true 
for any finite 6. In fact, by Carlson’s lemma,> which states that a 
band-limited function having a bandwidth less than 7 is uniquely 
determined by its sample values taken at integers along a half line, it 
follows that if go = 1, then (7? |g,| 4 0. But Carlson’s lemma does 
not say that }°7|g,| cannot be made arbitrarily small under these 
conditions. Lemma 1 (see below) shows that 77° |gn| can be arbi- 
trarily small. Thus, the right member of (25) is infinity, implying the 
truth of Theorem 1. 

An immediate corollary of Theorem 1 is that for the singular case 
of Szego’s theorem [f(@) vanishing on an interval] the infimum value 
of zero cannot be approached without using unbounded coefficients. 


Lemma 1: Let g(t) [not identically zero and € Ly(—~, ©)] have 
Fourier transform G(6) supported on (—pza, pr) for some fixed p, 0 < p 
< 1. Denote the samples of g(t) at the integers by gn Las in eg. (26) ], and 
fix the normalization of g(t) by setting |go| = 1. Then 


inf >> |gn| = 0, (28) 


where the infimum is taken over all g(t) having the indicated properties. 


Proof: We begin with the simple, but crucial, remark that it is suffi- 
cient that there be, for any p, a function h(t; p) € L?(— “, «~) whose 
Fourier transform is supported on (— pz, pz), such that h(0, p) = 1 
and such that 0? |ha(p) |? can be arbitrarily small.i This is sufficient, 


t We are grateful to H. J. Landau for pointing this out. Landau has also supplied 
an independent proof of the above refinement to Carlson’s lemma, which we give in 
the appendix. 
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because to make (27) large (for some fixed value of p) we would just 
need to take 


g(t) = m(s e) (29) 


for an appropriate A(t, p/2). Clearly, g(é) is band-limited to p and is 
[?(— ©, ©) because A(t, p/2) is bounded: 


p 1 pm /2 1 p7/2 
(t e) = — J H(@)d6 < se(or- f | H (6) at). (30) 
2 2a —pr/2 Qa —pr/2 
But can we really find an appropriate A(t) such that 


ho=1, > |hal? <«, (31) 
1 


or, equivalently, can we find a real h(t), band-limited to (—pz, pz), 
such that 


(i= Dra ye St (32) 
1 


Indeed we can, and in fact the answer may be extracted from an 
article by Salz® which discusses mean-square decision feedback equali- 
zation. Salz, in Section V of his paper, considered the equalization 
problem for faster-than-Nyquist signaling. His minimization problem 
was of the form in (32) plus an added term for the noise variance; 
h(t) corresponds to the output of the equalizer when one pulse of the 
form sin prt/prt is the input. He remarks, in the last sentence on 
page 1354 of his paper, that the quantity that corresponds to (32) 
plus added output noise variance goes to zero as the input noise 
variance decreases. Hence, if we choose h(t) to be the output pulse 
of a decision-feedback equalizer whose taps have been optimized for 
the case of sufficiently small input noise, then (82) will be sufficiently 
small. Thus, Lemma 1 is proven. 

The second question we discuss in this section is the rapidity with 
which the minimum distance decreases as p approaches zero. We 
develop this in Theorem 2. 
Theorem 2: 

2 
lim dinin(0) = 0 foranyk> 0. (33) 
p—0 p 
Proof: The proof is a simple construction. Consider the polynomials 
L 
Pi(z) =TI 0 — 274. (34) 


t=0 


Clearly, Pz(z) has a zero of order (2 + 1) at z = + 1, and has +1 
coefficients, with P,(0) = + 1. Now, for small p, the (LZ + 1)st order 
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zero at z = 1 implies 


3 [ | Pr (e) [2do = O(p?4+8) (35) 
ae 


for all integer L. Equation (33) follows immediately. 

Short of finding d?,,, exactly, there are a few mathematical questions 
that suggest themselves and that may be less difficult than the full 
problem. Thus, Fig. 1 prompts one to ask if there is a neighborhood 
of p = 1, where d3,,/4 is the pulse energy? Another question has to do 
with pulse design. Given that G(@) is symmetric, positive, L., and 
supported on (— pz, pw), is G(@) = constant the best choice to maximize 
the minimum distance (subject to fixed pulse energy)? 
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APPENDIX 
Landau’s Proof 


In Section IV we present another proof that 


95 2 
sup 3 = ®, (36) 
x Ign? 


where the sup is taken over all g(t) € L?(— ~, ), which are band- 

limited to (—pz, pr). Our proof in the text relied on the published 

results of work by Salz.6 Here we give a self-contained, but more 

mathematical, proof of (86) which was developed by H. J. Landau. 
Suppose (86) is not true, i.e., suppose that 


eo 2 
© |gnl? = 2 > 0 for all g(¢) of BW = pz. (37) 
1 

Then, 


lgol? SEE lanl. (38) 


From Carlson’s lemma, go is a linear functional on the 1, sequence 
{91) 92) °° +) 9x, ++} and, from (38), this linear functional is bounded. 
Therefore, by the standard Riesz representation’ for bounded linear 


tT Not all 1, sequences {g;} give rise to an appropriate g(t), and hence, the linear 
functional go is not defined on all of ls. Therefore, before using the Riesz theorem, 
the Hahn-Banach theorem should be invoked to extend go to a bounded linear func- 
tional on all of le. 
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functionals, we may write 
go= Ldn Lb? < (39) 
where the b, do not depend on g(t). We now consider the function 
p(z) =1— Y baa", (40) 


which is analytic for |z| <1. For any G(6) € L.(—pz, pr), we may 
write, using (39), 


i *" Goda = <b, [” em G(o)ds 
—ptT 1 —pT 
a : ‘“ ( ; baci Ghd, (41) 
—pr 1 
Therefore, 
lim = (1 a ba" \G)ad 0 (42) 
z{—-1 —pr 1 


for all G(@) © L2(—pz, pr).' By the completeness of Lz, we must have 
1 — oP bane’ = 0 a.e. on (—pz, pr). Since the radial limit of the H» 
function p(z) vanishes on a set of positive measure, p(z) itself must 
vanish for |z| <1. (See Ref. 7, p. 373, Theorem 17.18.) However, 
p(0) = 1, and, hence, we have a contradiction, denying the validity 
of (37). 
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Single-Integration, Adaptive Delta Modulation 


By P. CUMMISKEY 
(Manuscript received March 11, 1975) 


An estimate of optimum performance 1s derived for a single-integration, 
adaptive delta modulator. Several simulations of adaptive delta modulators 
with single integrators have all produced signal-to-noise ratios near or 
below the estimate. 

The derivations presented here indicate that the performance of a single- 
integration delta modulator is dependent on the correlation between adjacent 
samples of the input signal and on the probability density function of tts 
derivative. The relationship between the probability density of the derivative 
of the input signal and optimum performance, in turn, explains why 
signal-to-noise ratios taken on sine waves are greater than those recorded 
while processing speech signals. 


|. INTRODUCTION 


In this paper, an equation is derived for the optimum signal-to-noise 
ratio (s/n) of a single-integration, adaptive delta modulator. Mean- 
square quantizing noise is a mathematically tractable quantity which 
appears to be a reasonably good measure of overall performance. It 
was felt that an understanding of the relationships between this 
quantity and the character of the input signal would be useful. The 
derivations and data presented here all contribute to this end. Other 
practical considerations, such as subjective evaluation,! transmission 
errors,” and tandem encoding,’ have been discussed elsewhere. 

Several simulations? of single-integration, adaptive delta modulators 
on a variety of speech signals have produced s/n’s near or below the 
performance estimate suggested in this paper. It is further suggested 
that this estimate is very close to the upper bound on the performance 
of such coders. The s/n formula also provides an explanation of the 
disparities between s/n’s taken on sine waves and those obtained 
while coding speech signals. 

A block diagram of a single-integration, adaptive delta modulator 
is shown in Fig. 1. At the encoder, the difference between an input 
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Fig. 1—Single-integration delta modulation. 


sample, z;, and the previous output sample, £,-1, is quantized to one 
of two levels and coded. The code symbols, sgn (6;) through sgn (6; y) 
(where N may be any positive integer) are then interrogated by the 
companding logic, and the step size, A;-1, is altered before the ith 
sample is encoded. The quantized approximation to the difference, 
6; = A; sgn (6;), is added to the previous output to obtain the present 
output sample. 

The decoder operates in the same manner as the encoder except that 
the circuit is driven from the transmission channel rather than from a 
local comparator. The quantized signal at the decoder, 4;, is low-pass 
filtered to eliminate noise components outside the band of 2; (Le., 
frequencies greater than frp), and a replica of the input signal is thus 
regenerated at the desampling filter. 

The signal-to-noise ratios referred to in this paper were taken in 
the following manner. First the noise was obtained as shown in Fig. 2 
and then the ratio of input signal power to noise power was taken. The 
technique used by DeJager for sine wave s/n’s is described in Ref. 5. 
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Fig. 2—Quantizing noise measurement. 


Il. EXACT S/N FORMULAS 


The following equations are derived from the diagram in Fig. 1. 


6: = 2 — £1 


t= fac oe 


If the quantizing error is defined as 


A 


€; = Vi — Xi, 


(1) 
(2) 


(3) 


then, from (1) and (2), the following relationship holds: 


A 


Ee, = 0; —= 6;. 
From (8), it can be concluded that 


L, = «2; + 4, 
and likewise that 


tray = Bir Hee 


Therefore, (1) may be rewritten as 


6p A = Ci 


(4) 


(5) 
(6) 


(7) 


The average power in the prediction error is therefore 


E(8i) = E(x) + E(ai-1) + E(e-1) 


= 2H (xiti_1) = 2E (xe;—~1) + 2K (a 3_1€:-1), (8) 


where the # functions are expected or average values. It is now noted 
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that, for quasi-stationary signals, 


E(x) = E(a_1)," (9) 
and that 
E(é_1) = E(e).* (10) 


Therefore, eq. (8) may be reduced to 








E(& ’) = E(x, 4-1) a Ei (2@:-1) E (a i~1€;-1) | E (e?) : 
Be 72 [~“peh - wey trey tee OD 
The s/n at the quantizer is given as 
E (8) _ ES) 
= i 
s/ne EL(: 3 53)? ] ~ B(e) ( ) 
The s/n before filtering is defined as 
E(x) 
s/n = Ee)" (13) 
Note that (11) is equal to (12) divided by (13) or that 
a ( §2 
(67) _ 8/ng (14) 





E(2) s/n 


Hence, by substituting into (11) and transposing terms, an equation 
for the unfiltered s/n is obtained. 


s/ng — 1 
201 — CE (ai) /E (2) — LE (xen) 1/E (2) 
+ LE (ai-1e:-1) J/E (2) ]} 


Ill. ASSUMPTIONS AND APPROXIMATE FORMULAS 


s/n = (15) 


The variance of the prediction error is unknown because 6; contains 
quantizing noise [see (7) ]. Therefore, 6; cannot be optimally quantized. 

No meaningful information can be obtained directly from eqs. (1) 
through (15) without making some approximations or assumptions 
about the unknown terms [s/ng, E(a.e:-1) and E(a;_1¢:_1) |. Several 
measurements and simulations taken by the author and others before 
him support the following assumptions. 


(c) The optimum step size will yield the same signal-to-noise ratio 
at the quantizer that can be achieved by quantizing the noise- 
free part of 6; (i.e., the derivative of the input signal, x; — 2;_1). 


“To the extent that (9) and (10) are equations, (15) may be called an equation. 
Some awkward anomalies exist with regard to eq. (15); however, none of these is 
relevant to the problem. 
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(zz) The quantizing noise is the same as that generated by optimum 
quantization of (x; — 2;_1), and therefore 


ET (a; = Li_-1)€:_-1 | = 0. (16) 
Hence, 
E (x ei_1) = E(aj_1e;-1) = 0. (17) 


Given the above assumptions, (15) reduces to 


(s/Ngop — 1) 


8/0 = 3 Ea) VEG)’ oe 


where S/Ngop is the s/n achieved when x; — 2;_; is optimally 
quantized. 

(222) Finally, in an optimum modulator the quantizing noise spec- 
trum is flat. Then the ratio of overall noise to the inband noise 
is equal to the ratio of half the sampling frequency to the band- 
width of the input signal. 


Hence, the s/n taken on the filtered signal, ¢zp, is equal to the 
unfiltered s/n multiplied by the ratio of half the sampling frequency 
to the cutoff frequency of the filter. 


[s/neor — 1] 57 | 


s/Nyp = afr - Bee | 





(19) 


where f, is the sampling rate and frp is the cutoff frequency of the 
desampling filter or the bandwidth of the input signal. 

Equation (19) is identical to Nitadori’s signal-to-noise equation® 
for differential pcm. Nitadori cautions against its use in cases where the 
quantization is coarse, however. In this paper, eq. (19) is derived using 
somewhat different assumptions which, in fact, do appear to hold for 
delta modulation. 

The validity of the three assumptions given above is the main point 
of this paper. When these assumptions hold, an important relationship 
between the amplitude distribution of the derivative of the input signal 
and s/n performance can be drawn. 


IV. RELATIONSHIP BETWEEN S/N,.,» AND PROBABILITY DENSITY 
FUNCTION OF Xi Xt-1 


Paez and Glisson,’? among others, have shown that the amplitude 
probability distribution of speech and its derivatives is closely ap- 
proximated by the gamma distribution. Figure 3 shows that this dis- 
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Fig. 3—The amplitude probability density function of (zr; — z;-1) as compared 
with the Laplacian and gamma distributions. 


tribution closely approximates the probability distribution of 
(x; — x1) for telephone signals used in my simulations. The dis- 
tribution of (#; — x;~1), taken on the speech used in Jayant’s simula- 
tions, lies closer to a Laplacian distribution, however. 
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Table | — Signal-to-noise ratios of two-level quantizer output 


Probability Density Function B/ igus 
3t exp (— v3|y|/2c) 
Gamma PG y= eS ee 1.50 
V8z7r0|y| 
: 4 ao - v2 |y| ) 
Laplacian Py) = Wo exp ( os 2.00 
. exp (—y?/20?) 
Gaussian Pty) = ———=""— 2.75 
: ine 
1 A A 
Rectangular Pty) = 7 <y< 3 4.00 
A A 
PY) =U =o Uo 
Sinusoidal ‘ 
y = cos PGs = SG SI 5.28 
or . rvl a y? =y¥s 
y = sin @ 





Given a distribution that is symmetrical about the origin, the 
quantization step is optimum when 


[ v- aPway = 0, (20) 


where y relates to (x; — 2:_1). With the step set at the optimum size, 
Paez and Glisson, Max,® and others have calculated the noise power at 
the output of a two-level quantizer, 


BL — y= 2 f° w— 4)°Pwdy, (21) 
and achieved the s/n’s shown in Table I. 


V. COMPARISONS WITH SIMULATIONS 


The correlations between adjacent samples was taken on speech 
obtained using a carbon-button, telephone transducer. Similar data 
were obtained by N.S. Jayant on speech recorded from a high-fidelity 
transducer. Both signals were processed by Jayant’s adaptive delta 
modulator with a one-bit memory, where the step size is multiplied 
by 1.5 if the present and previous code words, sgn (6;) and sgn (6;-1), 
are alike, or by 0.66 if they differ. In all the simulations, the sampling 
and desampling filter cutoff frequencies are set at 3.3 kHz, except for 
the telephone speech recorded at 24 kHz. In this case, the cutoff was 
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reduced to 3 kHz. The telephone signals sampled at 48 kHz were also 
encoded by a single-integration delta modulator designed by D. E. 
Blahut.? Blahut’s encoder also performs close to the estimate [eq. 
(19) ], when processing telephone speech. Among the numerous coders 
tested, no single-integration delta modulator was found that performs 
significantly better than Blahut’s or Jayant’s. 

In Table II, performance estimates based on eq. (19) are compared 
with the s/n’s obtained using Blahut’s and Jayant’s delta modulators. 
To account for difference in probability density functions (see Fig. 3), 
the estimates were made with s/n,o, equal to 1.5 for telephone signals, 
and 2.0 for high-fidelity signals. 

The performance estimate given by eq. (19) is within 3.3 dB of the 
s/n’s obtained in simulations with Jayant’s delta modulator. The 
s/n’s taken on Jayant’s and Blahut’s coders, while processing telephone 
speech at 48 kHz and 24 kHz, are essentially equal to the estimate. In 
these cases, the signal level was carefully adjusted until optimum per- 
formance was obtained, then further data were taken to verify eq. (19). 
(See Table III.) 

The results shown in Table III lend great support to the approxima- 
tions made in deriving eq. (19). The noise terms do effectively cancel, 
leaving a residue that is at least an order of magnitude smaller than 
the noise-free terms in the denominator of (15) (see lines 5 and 6 in 
Table III). The estimates for noise rejection at the desampling filter 
and for quantizer performance (s/nq) are within 0.8 dB of the figures 
obtained in the simulations. 

Both coders were simulated with a 60-dB range of step sizes, and 
both were started with the step size equal to the minimum and the 


Table Il— Performance estimates 





Delta Modulator 





ae Sampling E(xjt:-1) | Estimate 10 logio Performance (dB) 
20R Rate (kHz) E(x?) (s/nup) (dB) 
Jayant’s | Blahut’s 
2.0 20 0.989 21.3 18.0 — 
*1.5 24 0.957 13.77 14.5 — 
2.0 40 0.997 30.0 28.0 — 
*1.5 48 0.9897 22.6 22.9 22.7 
2.0 60 0.999 36.5 34.0 — 


*Telephone speech: The acoustic-to-electronic response of the new 500-type, 
stations sets! indicates that signal components in the 100-Hz to 3.3-kHz band are 
differentiated, and that components below 100 Hz are severely attenuated. Hence, 
correlation between adjacent samples is lower for telephone speech than for high- 
fidelity speech. 

T At 24-kHz sampling, fre = 3 kHz. 
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Table Ill — Verification of eq. (19) 





om Coder Performance 
Fre- Estimates 
quency 
(kHz) Jayant’s Blahut’s 
s/NLp 24 13.7dB 14.5 dB — 
48 22.6 dB 22.9 dB 22.7 dB 
s/n 24 5.92 5.95 — 
(i.e., 7.72 dB) | (i.e., 7.75 dB) 
48 24.3 24.3 26.1 
(i.e., 13.9dB) | Ge. 13.9dB) | (ie, 14.2 dB) 
Noise rejection at the 24 6.0 dB 6.8 dB — 
desampling filter 48 8.7 dB 9.0 dB 8.5 dB 
= 10 logio(fs/2fiP) 
s/ng 24 1.5 1.531 — 
48 1.5 1.555 1.569 
[1 _ E(wxiti-s) 24 — 0.0422 — 
E (2?) 48 — 0.0103 0.0103 
Ei (a:e:-1) 24 0 — 0.00238 —_ 
E (2?) 48 0 —0.00112 — 0.00060 
_ E (x:~1€;-1) 
E (2) 


predictor voltage equal to zero. As the average input signal level was 
varied over a 40-dB range, it was found that the s/n varied by 3 dB. 
In either coder, it was found that when performance fell significantly 
below the estimate (19), the following phenomena were observed : 


(t) Quantizer performance and unfiltered s/n changed slightly 
(in some cases, these parameters increased in value). 

(21) The noise terms no longer effectively canceled. 

(27) There was a dramatic reduction in noise rejection at the de- 
sampling filter. It appears that when the correlation between 
the difference signal, (#; — x;-1), and the noise (16) becomes 
significant, more noise must shift into the passband of the 
desampling filter. 


Hence, the approximations used in deriving eq. (19) do appear to 
describe the optimum condition. 

These results have been obtained using both an HP2100A mini- 
computer and an IBM 370, and therefore are repeatable. Moreover, 
further validation by others using other encoders is desirable. 
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Vi. SINE WAVE PERFORMANCE 


Another interesting check on the theory is the fact that it explains 
why researchers everywhere achieve much higher s/n with sine wave 
inputs than with speech signals. DeJager’s formula [see eq. (22) ] 
indicates that the s/n taken on a sine wave at any frequency below 
3 kHz is greater than the s/n that we predict or obtain for telephone 
speech. 

fs 
8/NDeJager = (0.04) f?- fp ? (22) 
where f is the frequency of the input sine wave. 

The amplitude probability distribution and s/n,,, for a sine wave 
were given in Table I. Substitution of the value in Table I into eq. (19) 
yields an estimate for sine wave s/n’s. 





4,28 Je 
S/Nsine wave = ye, (23) 


Equation (23), in turn, is approximately equivalent to DeJager’s 
formula. This relationship can be shown as follows. Let x = sin (27 ft); 
then 


(weit) _ i? [sin (2x ft) ]-sin (2a ft + 2af/f.)dt 





nee 1 mel 
E(«) if sin? (Qu ft)dt 
or 
Bees) ath (77). (25) 


If the delay angle, (27f/f,.), is sufficiently small, then 
2 f2 
2[1 cos (72) | x1 — cost (792) a EL. (26) 


When (26) is substituted into (23), we obtain something very close 
to DeJager’s formula: 





3 
Pie 
For f = 800 Hz, fre = 3.3 kHz, and f, = 48 kHz, estimates of 33.5 


and 34.7 dB are obtained using (22) and (27). Under these same con- 
ditions, signal-to-noise readings of 26 to 27 dB were obtained in simula- 


S/Nsine wave — (0.054) 





(27) 
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tions of Jayant’s delta modulator. Under similar conditions, DeJager® 
obtained a maximum s/n of about 30 dB on a linear delta modulator. 


Vil. CONCLUSIONS 


The optimum performance of Blahut’s and Jayant’s delta modulators 
is very close to the estimate, (19), when processing speech signals. 
Further experimentation with step-size compandors, without a change 
in the prediction technique, will not produce significantly higher 
signal-to-noise ratios. Equation (19) applies to a delta modulator with 
a single, ideal integrator; therefore, it does not preclude improvements 
through the use of fixed, higher-order networks. 

In addition, it has been shown that delta modulator performance is 
dependent on the amplitude probability distribution of the derivative 
of the input signal. This dependence should be tested on a variety of 
signals and probability density functions. The theory also implies that 
a relationship exists between the amplitude distributions of differ- 
ential waves at the input and optimum s/n, when higher-order net- 
works are used. 

Finally, I wish to call attention to the fact that the s/n performance 
of a delta modulator is significantly less for telephone signals than 
for low-pass filtered, high-fidelity signals, or for sine waves. 
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A Low-Bit-Rate Interframe Coder 
for Videotelephone 


By B. G. HASKELL and R. L. SCHMIDT 


(Manuscript received January 7, 1975) 


It has been suggested that customers for videotelephone service may be 
more interested in graphical information and in views of stationary objects 
than in head-and-shoulder views of people engaged in conversation. For 
this reason, an interframe coder simulation was constructed of a system 
that transmits graphics with full seven-bit PCM resolution, but displays 
scenes containing much movement with visible smearing in the moving 
areas. 

With the coder operating at 200 kb/s (0.1 bit per pel for a 1-MHz 
signal), a very usable (somewhat reduced-resolution) graphics picture 
can be transmitted in about one-half second, which is about as fast as the 
human eye can assimilate the information. A full-resolution picture ts 
built up after 3 to 5 seconds but, except for high-detail scenes, it 1s very 
difficult to tell the difference between the half-second picture and the 5-second 
picture. 

Head-and-shoulder views of people engaged in low-key conversations 
are transmitted with quite adequate picture quality. Moving lips appear 
somewhat smeared, but it may not be enough to be objectionable if the audio 
is suitably delayed. However, large area movement is very vistbly smeared— 
even to the point of being unrecognizable at moderate speeds. Whether or 
not this feature makes the coder unusable depends upon the value the 
user places on high-quality animated face-to-face conversation. 

Briefly, the coder works as follows: First, the signal is temporally pre- 
filtered. Then moving-area pels are sent as line-to-line differences of frame- 
to-frame differences. As the buffer fills, field-to-field, pel-to-pel, and frame- 
to-frame subsampling as well as adaptive quantization are brought in as 
needed to reduce the data rate. 


I. INTRODUCTION AND SUMMARY 


The use of videotelephone for graphical information and for views 
of stationary objects has profound implications in long-distance trans- 
mission of video signals via frame-to-frame coding, where the required 
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data rate is directly dependent on the amount of movement to be 
accommodated in the scene. 

If it can be shown that visible degradation of moving areas in a 
television picture is not detrimental to the effectiveness of visual 
communication, then a significant saving in transmission costs is 
possible with frame-to-frame coding.! With these techniques, stationary 
areas of pictures would be transmitted with full resolution, while 
moving areas would be sent with visibly reduced resolution. 

Transmission of graphics or still pictures can be accomplished in a 
particularly pleasing way, subjectively speaking. A reduced-resolution, 
but quite recognizable, picture appears very quickly at the receiver. 
Full resolution is then built up over a period of time that depends on 
the transmission channel data rate. However, for the majority of 
pictures, it is difficult for an observer to tell the difference between 
the full resolution picture and the earlier-appearing reduced-resolution 
picture. In this regard, such a system would be much more usable for 
interactive visual communication than would a facsimile or slow- 
scan system operating at the same data rate where a complete picture 
would not be visible for a relatively long time. Also, interframe coding 
can handle small amounts of movement such as adding a few lines to a 
sketch or using a pointer with stationary graphics, whereas a slow- 
scan system would be very unsatisfactory. 

Scenes of people engaged in conversation do not fare as well as 
scenes in which there is little or no movement. Moving areas such 
as a person’s lips and eyes are visibly smeared and, depending on the 
data rate, large-area movement is jerky because of coder overload. 
Even so, a well-behaved subject can present a very decent picture 
to the receiver if he or she is aware of the limitations of the medium. 
However, it is this aspect of low-bit-rate interframe coding that raises 
questions in most people’s minds. Whether or not this feature makes 
such techniques unusable depends upon the value the user places on 
high-fidelity, animated, face-to-face conversation. 

To move closer to the answers to some of these questions, an inter- 
frame coder simulation was constructed for 1-MHz videotelephone 
signals that was designed to operate in the hundreds of kilobits per 
second range (below 3 bit per picture element). Many techniques are 
used to adaptively reduce the moving-area resolution (both spatial 
and temporal) in proportion to the amount of motion, and to restore 
full resolution to the display as quickly as possible after motion ceases. 

With such a system operating at 200 kb/s (0.1 bits/pel), a recog- 
nizable, somewhat reduced-resolution graphics picture is displayed at 
the receiver in about one-half second. Full resolution requires 3 to 5 
seconds. With head-and-shoulder views of people engaged in low-key 
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conversation, moving lips appear somewhat smeared, but this may not 
be enough to be objectionable if the audio is suitably delayed. How- 
ever, large-area movement is very visibly smeared and jerky, even 
to the point of being unrecognizable at moderate speeds. 

With the system operating at 50 kb/s, a reduced-resolution graphics 
picture requires about 2 seconds for transmission, while full resolution 
takes 10 to 15 seconds. At 50 kb/s, face-to-face conversation loses much 
of its naturalness. Lip motion can be followed only if the subject 
remains otherwise absolutely still, and large area motion is portrayed 
as a series of snapshots occurring at a rate of about 1 per second. It is 
interesting to note, however, that, even at 50 kb/s, useful interactive 
visual communication is still possible using interframe coding whereas, 
with slow-scan operating at the same data rate and requiring about 
10 seconds per frame for transmission, interactive communication is 
severely hampered. 

In the following sections, the technical aspects of the coder and the 
simulation are discussed. 


ll. MULTIMODE CONDITIONAL REPLENISHMENT 


It is well known that, in a television signal, successive frames are 
very much alike. The frame-to-frame differences are negligibly small 
except in areas of the picture that contain moving objects. Thus, if 
frame memories are provided at the transmitter and receiver of a 
video communication system, it is necessary only to transmit those 
areas of each frame where the frame differences are significant. The 
remaining picture elements (pels) can be repeated from the previous 
frame. This technique is called conditional replenishment.? Conditional 
replenishment requires addressing the pels which are transmitted 
(the changed pels or ‘‘moving-area”’ pels) and buffers at the trans- 
mitter and receiver. 

For example, in Ref. 3 a conditional replenishment coder for eight- 
bit pcm videotelephone signals* is described which operates at 2 Mb/s 
(one bit per pel on the average) and uses a number of techniques to 
reduce the bit rate required for transmission. The pels to be trans- 
mitted are addressed along the line in clusters, and their amplitudes 
are sent as frame-to-frame differences. When the transmitter buffer 
starts to fill, indicating active motion, only every other changed pel is 
transmitted,*:+ with the unsampled pels being replaced by the average 
of their neighbors. When the buffer fills completely, replenishment is 
stopped for one frame period, allowing the muse to empty before 
resuming transmission. 


* 30-Hz frame rate, 271 lines, 2:1 interlace, 3 dB down at 1 MHz, 2-MHz sampling 
rate, 8-bits/sample, 210 visible samples/line. 
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Other multimode conditional replenishment coders are described in 
Refs. 5, 6, and 7. A variety of techniques control the rate of data 
generation to prevent buffer overflow. 

Other functions of conditional replenishment coders, such as the 
sending of synchronizing information and the accommodation of 
transmission errors, are also discussed in Refs. 1 to 7. 


Il. LINEAR PREDICTIVE CODING 


A linear predictive coder forms a prediction of each pel to be sent 
by computing a linear combination of previously transmitted pels. 
The difference between the actual value and the prediction is then 
quantized, coded, and transmitted. The inverse process takes place 
at the receiver. The better the prediction, the smaller the entropy 
of the differential signal and the bit rate required for transmission. 
Figure 1 shows two successive frames with interlacing assumed (two 
interlaced fields per frame). Suppose Z is a moving-area pel we wish 
to transmit. Pels A, B, C, G, and H are in the field presently being 
scanned; pels D, E, F, R, 8, and T are in the previous field; and the 
remaining pels are one frame period back from the present field. Pel M 
is the previous frame value of Z, and if it is used as a prediction of Z, 
then Z — M, the differential signal which is transmitted, is the frame 
difference as discussed above. 

In Refs. 8 and 9 it was found that using M + (B — J) as a pre- 
diction of Z resulted in a relatively-low-entropy, differential signal 
compared with other nonadaptive predictive coders. In this case, 
the transmitted differential signal is the line-to-line difference of the 
frame-difference signal (Z — M) — (B — J). 

Transmitting line differences of frame differences has several other 
advantages as well. Since it does not use pels along the present line 


PREVIOUS FRAME PELS 





PREVIOUS FIELD PELS 


Fig. 1—Two successive television frames, interlacing assumed (two interlaced 
fields per frame). Pels Z and M are exactly one frame apart. 
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Fig. 2—Predictive coder which transmits only the moving-area pels. The differ- 
ential signal (Z-M)—(B-J) is the line-to-line difference of the frame-to-frame differ- 
ence. The segmenter (not shown) determines whether or not Z is a moving-area pel. 


If it is, the switch is put in the up position and Z, a new quantized value, enters the 
frame memory. Otherwise, the switch is put in the down position and the previous 


frame value M is recirculated. In any event, Z is the value displayed at the receiver 
in the absence of transmission errors. 


or pels in the previous field for its prediction, pel subsampling and field 
subsampling can be employed without affecting the performance of 
the predictor. Also, it has been found that relatively few quantization 
levels are required to produce a good quality picture. Starting with 
seven-bit pcm, 1l-level quantization* of the line difference of frame 
difference is sufficient for most pictures and most speeds of move- 
ment, whereas 30- to 40-level quantization is required for the frame- 
difference signal. 

Figure 2 shows a single-mode conditional replenishment coder which 
transmits quantized line differences of frame differences in the ‘““moving 
area.”’ This is the predictive technique used in the coder described in 
this paper. As with all conditional replenishment coders, a “‘segmenter”’ 
is required to divide the picture into moving parts and stationary 
parts,®.7 logic must be provided for sending addressing and synchro- 
nizing information, and a buffer is needed to smooth the data rate 
prior to transmission. If Z is a moving-area pel, the switch is in the 
up position to allow the quantized representation Z to pass through 


“On a scale of 0---127, the quantization levels are 0, +1, 43, +10, +23, +48. 
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to the frame memory. If Z is a stationary-area pel, the switch is in the 
down position, and the previous frame pel M is repeated. At the 
receiver, the inverse process takes place, and the value Z is displayed. 

To take advantage of the low entropy of the line difference of frame 
difference, a variable word-length coder should be used to code the 
quantized moving-area differential signal. A suitable code for 11-level 
quantization is given in Table J. The four-bit code word 0000 is 
reserved for signaling the end of a cluster of significant changes.® 
In a later section of this paper, nine-level quantization is discussed. 
The first nine code words of Table I are suitable for nine-level 
quantization. 


IV. TEMPORAL FILTERING 


A simple method of reducing the data rate in an interframe coder 
for television pictures is to subsample in the temporal direction and 
transmit only every other frame (odd field followed by even field) 
which enters the coder, i.e., send frames at a rate of only 15 Hz. At 
the receiver in place of each missing frame, one would display either 
the previous frame or an interpolation of the previous frame and the 
upcoming frame. However, when using this technique Jerkiness is 
visible in the displayed picture for all except the very slowest 
movement. 

The jerkiness is due to aliasing in the temporal-axis frequency 
domain, i.e., the input signal has significant power above the half- 
sampling frequency (here, 7.5 Hz). Aliasing can be reduced by filtering 
the input signal to reduce as much as possible the power above 7.5 Hz 
in the temporal frequency domain. Instead of jerkiness, the displayed 
signal then exhibits blurring in the moving area in proportion to the 
speed of movement. Many viewers find this type of distortion prefer- 


Table |— Variable word-length code suitable for 11-level 
quantization with code word 0000 reserved for indicating 
the end of a cluster of significant changes 





Lo 1 

+L, 01 

—L; 001 
+L, 0001000 
—-L, 0001001 
+L; 0001010 
—L; 0001011 
+L, 0001100 
—L, 0001101 
+L; 0001110 
—L; 0001111 
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AVERAGING 
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_— (b) FRAME AVERAGING 


POWER TRANSFER CHARACTERISTICS 





0 75 
TEMPORAL FREQUENCY IN HERTZ 


Fig. 8—Power transfer characteristics versus temporal frequency. (a) Light 
integration by the camera alone. (b) Simple frame averaging. (c) Averaging plus 
frame repeating as shown in Fig. 4. 


able to jerkiness, since it is already present to some degree in all 
television pictures. 

Ideal low-pass filtering using a (sin x)/xz impulse response filter 
would require several frame memories. In this paper, we use a method 
of temporal filtering employing only one frame memory, namely, the 
one normally present in the interframe coder. 

Some temporal filtering already takes place in a normal television 
camera because of its integrating action. Figure 38a shows the power 
transfer characteristic (derived in the appendix) owing to integration 
of the light falling on the camera target. 

Additional temporal filtering using a frame memory can be carried 
out by a simple averaging of the incoming frame and the previous 
frame. The power transfer characteristic of this type of filtering 
(derived in the appendix) is shown in Fig. 3b. It is down by about 8 
dB at 7.5 Hz. 

Figure 4 shows the implementation of frame repeating plus temporal 
averaging. The switch is held in the down position during alternate 
input frames. Otherwise, it performs conditional replenishment under 
control of the segmenter as in Fig. 2. In this case, the ‘‘previous frame”’ 
coming out of the frame memory during conditional replenishment is 
not the previous frame at all, but, as a result of the frame repeating, 
it is actually the frame that was coded two frames ago. Because of 
this fact, increased temporal filtering occurs. Figure 8c shows the 
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Fig. 4—Implementation of predictive coding with temporal filtering and frame 
repeating. (a) Transmitter. (b) Receiver. During alternate incoming frames, the 
switches are held in the down position, thus recirculating the contents of the frame 
memory, and no data are fed to the transmitter buffer. 


power transfer characteristic of frame repeating plus temporal aver- 
aging (also derived in the appendix). It falls off much faster than 
curve b, and is down by about 10 dB at 7.5 Hz. However, unlike 
curve b it rises again at higher frequencies. 

Temporal averaging and frame repeating as shown in Fig. 4 has 
been implemented, and jerkiness is difficult to detect. However, 
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blurring is quite visible when the subject moves. Lip motion is also 
blurred somewhat. 

Temporal filtering helps to reduce the data rate in two ways. First, 
as already mentioned, only every other television frame need be 
transmitted. Simple frame repeating at the receiver is sufficient to 
display a picture in which jerkiness is difficult to detect. Second, the 
blurring of the moving area makes the signal more amenable to 
predictive coding. With the blurred picture, the differential signal is 
smaller on the average, thus reducing its entropy and the bit rate 
required for transmission. ° 


V. SEGMENTING, ADDRESSING, AND SYNCHRONIZING 


The coder uses simple, well-known techniques for segmenting the 
picture into moving and stationary areas.* Ordinary seven-bit PcM re- 
quires updating of pels which have changed by 2 or more on a scale 
of 0---127 to present good picture quality in slowly moving areas. 
However, temporal filtering as shown in Fig. 4 amounts to halving all 
the frame differences. Thus, in the coder described here, frame differ- 
ences larger in magnitude than 1 on a scale of 0---127 are detected 
and labeled as significant changes. As is described later, the frame- 
difference threshold is raised to 2 to reduce the data rate when buffer 
overflow threatens. 

Significant changes because of camera noise are dealt with as in 
Ref. 3. That is, a change is ignored if the two pels on the left and the 
two pels on the right have not changed significantly. 

Positioning information for the transmitted pels is also sent as in 
Ref. 3. The start of a cluster of significant changes is signaled by an 
elght-bit address indicating its position. The end of a cluster is specified 
by sending a four-bit code word which is distinguishable from the 
quantizer output code words (see Table I). 

Small gaps between clusters are more efficiently handled by trans- 
mitting the pels therein than by ending one cluster and starting a new 
one.’ This technique is called gap-bridging. 

In Ref. 8, it was found that the entropy of the quantized line differ- 
ence of frame-difference signal was somewhat above two bits per 
moving-area pel. Since each new cluster requires twelve bits for address- 
ing, the coder bridges gaps of six pels or less prior to conditional 
replenishment. 

Synchronizing is handled as follows. Since there are less than 256 
visible pels along a line, frame syne, field syne, error detection words, 
and other events which occur relatively rarely can be signaled con- 
veniently using eight-bit code words that are distinguishable from 
the eight-bit cluster addresses. However, line-to-line syne is not 
handled as easily. 
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If line syne were signaled with an eight-bit word, then, with the 
~ 8-kHz line rate used here, 64 kb/s would be devoted to line syne. 
For a coder operating at a few hundred kb/s, this is much too high a 
proportion of the total bit rate. 

The method of line syne proposed for the coder requires slightly 
more than one bit per line. With frame repeating, this amounts to 
about 4 kb/s being used for line syne. The method relies on the fact 
that the first pel in the first cluster of a line is usually located to the 
left of the last pel of the last cluster of the previous line. In this case, 
no additional information need be transmitted to tell the receiver 
that a new line has begun. However, the receiver must be told if the 
above situation does not apply, and it must also be told which lines 
in the picture contain no clusters. 


000000000000000000000000 
ABCDoooo00000000EFGH 
000000000000000000000000 
000000000000000000000000 
oolJ IX LooooM N Poooo0000 
000000000000000000000000 
00000000000000000000QR8S 


Consider the field of pels shown above. Pels labeled A, B, C, ---, 
R, S have changed significantly and must be transmitted along with 
their cluster addresses. Pels labeled o will not be transmitted. Since 
the cluster ABCD is the first one in the field, the receiver need not be 
told that a new line is starting. It only needs to be told the number of 
lines at the beginning of the field that contains no clusters. A string 
of zeros equal in number to this amount followed by a one suffices to 
convey this information to the receiver. For implementation reasons 
which become apparent later, this string of bits is transmitted after 
the address word of cluster ABCD and before the pels A, B, C, D and 
the end-cluster message are sent. Cluster EFGH is sent in the normal 
manner, i.e., address, pels, and end-cluster. 

Since pel I is to the left of pel H, the receiver can tell from the 
address of cluster IJKL that a new line has begun. Following the 
address word of cluster IJICL, the bits 001 are transmitted, indicating 
that two intervening lines contained no clusters. Cluster MNP is 
sent in the normal manner. 

Since pel Q occurs to the right of pel P, the receiver cannot tell 
from the address of cluster QRS that a new line has begun. A special 
reserved address word must be transmitted to indicate a new line. 
Following this, the address of cluster QRS and the bits 01 are trans- 
mitted as usual. If small gaps between clusters are bridged, then the 
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above procedure should be modified somewhat. In this case, the 
special reserved address word need be transmitted only if 


Q-address > P-address + minimum gap size. 
A system using these ideas would operate sequentially as follows: 


(z) At the start of each field 
(a) A field syne word is transmitted. 

(b) An address register is set to maximum value. 

(c) A counter is reset to zero. 

(iz) The counter is incremented by 1 at the end of each line which 
contains no clusters to be transmitted. 

(472) When the first cluster of a line is encountered 

(a) A check is made to see if the address of the first pel exceeds 
that in the address register. If it does, a special reserved 
eight-bit word is transmitted which is distinguishable 
from all the normal cluster address words. This should 
not occur very often when movement is significant. 

(b) The address of the cluster is transmitted. 

(c) A string of zeros is transmitted equal in number to the 
value stored in the counter. None are sent if the counter 
equals zero. 

(d) A one is transmitted, and the counter is reset to zero. 

(7v) Normal conditional replenishment then resumes and con- 
tinues until the end of the line. 

(v) The address of the last pel of the last cluster of the line is 
added to the minimum gap size, and the result is stored in the 
address register. 

(vt) Operation continues with Step (77). 


This technique was tested, and with scenes containing slow, 
moderate, or rapid movement the number of special words that had 
to be transmitted rarely exceeded two per field (0.5 kb/s when frame 
repeating is employed). With no movement, the clusters of significant 
changes resulting from noise occurred randomly, and the number of 
special words was higher. But in this case the overall data rate is very 
small, and thus the special words do not overload the coder. 


Vi. MODE CONTROL 


For a given transmission bit rate, a higher overall picture quality 
can be obtained if the coding is adapted to the amount of movement 
in the scene. For an interframe coder, the fullness of the transmitter 
buffer is the simplest and most useful measure of the amount of 
movement.'—* Imminent buffer overflow is a direct indication that 
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the data rate being generated is too high and that the displayed 
moving area resolution should be reduced. 

The basic operating mode of the low bit-rate coder is shown in 
Fig. 4, i.e., temporal filtering, frame repeating, and transmission of 
line differences of frame differences in the moving area. As with 
previous coders operating at higher bit rates,?3>—-’ the moving-area 
resolution is reduced by switching to a lower resolution mode if the 
buffer queue length exceeds some fixed threshold. Thus, as shown in 
Fig. 5, if the buffer queue length exceeds 7, then coding mode 1 is 
invoked; if it exceeds T2, then coding mode 2 is invoked; etc. Mode 4 
is frame repeating, i.e., the switch in Fig. 4 is held in the down position, 
and no data are generated except synchronizing information. In this 
way, buffer overflow is prevented. 

When motion in the scene ceases and the size of the moving area 
decreases, the buffer begins to empty, and a higher-resolution coding 
mode should be used. To prevent oscillations between coding modes, 
a higher-resolution mode is not invoked until the end of a field, and 
then only if the buffer queue length is below 7, for modes 1 and 2 
and TJ. for modes 3 and 4. Thus, for example, a change from mode 1 
to mode 2 is possible any time the buffer queue length exceeds T», 
but a change from mode 2 to mode 1 can occur only at the end of a 
field in which the buffer queue length falls below 71. 


Vil. MODES USED IN THE CODER 


Mode 0 is the previously mentioned basic operating mode shown 
in Fig. 4. An odd field and an even field are coded as shown in Fig. 6a. 
Then the next two fields are skipped; at the receiver, the frame is 
repeated by displaying the stored signal. Mode 0 is the highest resolu- 
tion mode of the coder. 


EMPTY FULL 





0 1 2 3 4 


CODING MODE 


Fig. 5—Switching between coding modes under control of the transmitter buffer 
causes the moving area resolution to be reduced as the amount of motion in the scene 
increases. Mode 0 codes with the highest resolution, mode 4 with the lowest. When 
the buffer queue length exceeds T;, mode 7 is invoked (except for mode 3 which is 
invoked at the end of the field). At the end of the field in which the buffer queue length 
falls below 7; for modes 1 and 2 or 72 for modes 3 and 4, mode 7 is revoked and mode 
t—1 is invoked. With this strategy, oscillations between coding modes are prevented. 
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(a) MODE 0 


UPDATE INTERPOLATE REPEAT INTERPOLATE UPDATE INTERPOLATE} REPEAT... 
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(b) MODES 1 AND 2 


Fig. 6—(a) Simple frame repeating is used in mode 0. Two fields are updated, then 
two fields are repeated. (b) Frame repeating and field interpolation are used in modes 
1 and 2. Only one out of four fields is updated. No data are generated for the remain- 
ing three fields. 


Mode 1 is interpolation of even fields. In this mode, the data rate is 
halved by not transmitting even-numbered fields as shown in Fig. 6b. 
Instead, an interpolation between the previous odd field and the 
upcoming odd field” is displayed, thus reducing the vertical resolution 
in the picture by a factor of two. 

Field interpolation is implemented as shown in Fig. 7. If, during 
input of an even field, mode 1 is invoked, then the conditional re- 


INTERPOLATION 







DISPLAY 
e 
e 


Fig. 7—Implementation of field interpolation. S: is held in the down position 
during input of repeated fields and interpolated fields. No data are generated for 
them. One field period later, S2 is put in the up position to display interpolated fields 
and in the down position to display updated and repeated fields (see Fig. 1). 
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plenishment switch S; is held in the down position for the remainder 
of the field, and no updating occurs. During input of the next two 
successive odd fields, switch 2 is held in the up position to display 
interpolated values for the even fields. Otherwise, it is in the down 
position. Display of two interpolated fields is necessary because of the 
aforementioned frame repeating which would otherwise display the 
invalid contents of the frame memory. 

Mode 2 consists of the field interpolation of mode 1 plus 2:1 hori- 
zontal subsampling,?* i.e., only every other moving-area pel along 
a line is transmitted. The untransmitted pels are obtained from their 
neighbors by interpolation. Subsampling reduces the data rate by 
a factor of almost 2 over mode 1. 

Mode 2 also employs coarser quantization of the line difference of 
frame-difference signal and an increase of the frame-difference threshold 
used by the segmenter. When mode 2 is invoked, the frame-difference 
threshold is raised from 1 to 2 on a scale of 0---127, and the two 
smallest nonzero levels of the quantizer are switched out of operation, 
reducing the number of levels to nine. The outputs of the nine-level 
quantizer are coded using the first nine code words of Table I. Coarser 
quantization reduces the entropy of the differential signal, and raising 
the frame-difference threshold reduces the number of pels that must 
be transmitted. Together they reduce the data rate by a factor of 
about 1.5, but this figure depends very much on the picture material 
and on the amount of movement in the scene. 

Mode 8 is frame repeating at the end of a field. When mode 38 is 
invoked, all conditional replenishment is halted. The contents of the 
frame memory are displayed for odd-numbered fields, and interpolated 
values are displayed for even-numbered fields. But unlike the other 
modes, it is invoked only at the end of a field. The purpose of this is 
to avoid the picture breakup associated with the stopping of condi- 
tional replenishment in the middle of a field. As the amount of motion 
in the scene increases, mode 3 causes the coder to progressively operate 
in 4:1 frame repeating, 6:1 frame repeating, or as much as is necessary 


Table Il — Modes of the coder with mode 0 the highest 
resolution mode 


Mode 


Temporal filtering and frame repeating. 

Mode 0 plus interpolation of even fields. 

Mode 1 plus 2:1 horizontal subsampling, increased 
frame-difference threshold, and coarser quantization. 

Frame repeat at end of field. 

Instantaneous frame repeat. 


Rw Nero 
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to accommodate the rate of data generation. Mode 8 is revoked at the 
end of the field during which the buffer queue length falls below T2 
(not 71, as with modes 1 and 2). 

Mode 4 is instantaneous frame repeating. It is rarely used and is 
invoked only to prevent data from being lost in an uncontrolled manner 
because of buffer overflow. It is revoked at the end of the field, and 
normal frame repeating under mode 3 then resumes. The modes of 
the coder are summarized in Table II. 


Vill. CHOICE OF BUFFER QUEUE LENGTH THRESHOLDS 


The objective of the coder is to operate in the mode that best 
matches the data generation rate with the channel transmission rate. 
Also, oscillation between modes must be avoided since it adversely 
affects picture quality in some cases. Correct choice of the buffer queue 
length thresholds is very important in accomplishing these objectives. 
As an example, the following illustrates how the thresholds might be 
chosen for a 200-kb/s channel rate. 

Mode 0 is used only when there is little or no motion in the scene. 
Its most important function occurs just after motion in the scene has 
ceased and mode 1 (interpolation of even fields) has been revoked. 
The objective is to update the even field and restore full vertical 
resolution as quickly as possible. Shortly after even field update has 
begun, the buffer queue length will exceed 7; and updating will cease. 
Little or no data will be produced for the remainder of the field and 
for the next three field periods. If during this time the buffer empties, 
then transmission time will have been wasted. Thus, 71 should be 
chosen large enough so that the buffer cannot empty in four field 
periods (1/15 second). For 200 kb/s, 71 > 133383 bits.* 

During mode 1, data are produced in only one field out of four 
(see Fig. 6b). If the overall data generation rate happens to equal 
the channel transmission rate, then the coder should not produce any 
data if it should switch to mode 0, and it should not switch to mode 2. 
In Fig. 6 at the end of field 1 coded in mode 1, the buffer queue length 
will exceed 71, and thus field 2 will be interpolated, field 3 will be 
repeated, and field 4 will be interpolated even if the coder drops into 
mode 0. To prevent mode 2 from being switched in during a mode 1 
odd-field update, 7: must be large enough to accommodate the 
accumulated difference between the data generation rate and the 
channel transmission rate. Somewhat more than three field periods 
of channel data may have to be buffered. Thus, for 200 kb/s, Tz > T1 
+ 10000 bits. 


“This figure can be halved if, in mode 0, fields 4, 8, --- (see Fig. 6) are updated 
instead of repeated. 
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With mode 2, the same sort of argument applies. Switching to 
adjacent modes when the data generation rate and the channel trans- 
mission are well matched can be prevented by separating the thresholds 
by more than three field periods of channel data. Thus, 7, and T; 
should be more than 10000 bits larger than the next lower threshold. 

Mode 38 (4:1, 6:1, ---frame repeating) is invoked at the end of a 
field in which the buffer queue length exceeds 7';. It is revoked at the 
end of a field in which the buffer queue length falls below JT. If the 
buffer queue length exceeds 7’; at the end of field 1 in Fig. 6b, then 4:1 
frame repeating will occur if the buffer queue length is still above T, 
at the end of field 4. This can be guaranteed by choosing T; — T» 
larger than three field periods of channel data. Thus, for 200 kb/s, 
T; > T, + 10000 bits. If 7, — 73 exceeds four field periods of channel 
data, then normal 6:1 frame repeating with no picture breakup can 
occur; if it exceeds eight field periods of channel data, then normal 8:1 
frame repeating can occur; etc. 

T, also determines the transmission delay owing to buffering. If 
300 ms is the maximum tolerable one-way delay,’ then 7’, must not 
exceed 0.3 X channel rate. For 200 kb/s, T, < 60000 bits. 

Table III gives suitable buffer queue length threshold values for 
200-kb/s and 50-kb/s operation. The distances (7. — 71) and (7,4 — 7's) 
can be reduced somewhat without seriously affecting coder operation, 
and other more complex mode control strategies can probably be 
devised that do not require as much buffering. But for purposes of 
assessing the possible trade-offs between picture quality and channel 
bit rate, these settings are a valid compromise. 


IX. DIGITAL TRANSMISSION ERRORS 


It seems to be a general rule that the more the redundancy in a 
stream of information is reduced, the more vital the remaining in- 
formation becomes. This is especially true for a low bit-rate interframe 
coder for television signals. Errors in the data which arrive at the 
receiver wili usually cause discrepancies in its frame memory of which 
the transmitter is unaware. Thus, if no means are provided to ac- 


Table |!]— Buffer queue length thresholds for 200 and 50 kb/s 
Threshold 200-kb/s Operation 50-kb/s Operation 
T 13400 3350 
T. 24900 6230 
Ts 45000 11250 
Ts 60000 15000 
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commodate for digital transmission errors, they will cause visible 
picture degradations that will last forever. 

Many schemes for handling errors have been suggested.!? A simple 
method, called forced updating, is to transmit a portion of the data 
as pom. After a period of time, data in the receiver frame memory 
will be corrected and visible errors will disappear. However, using 
channel bit rates which are relatively low compared with pcm, the 
time required for correction can be quite long. For example, if 10 
percent of the channel data is devoted to pcm, then at 200 kb/s all 
errors in the picture can be corrected in about 22 seconds. But with 
one error in 10° bits, for example, the average time between errors 
is 5 seconds. Thus, with this technique, errors are always present and, 
with differential coding of the type discussed previously, highly 
visible. Other error control techniques are obviously necessary. 

Randomly occurring, isolated errors and occasional bursts of errors 
can be dealt with fairly easily by lowering the information bit rate 
and using forward-acting error correction codes. However, long bursts 
of errors in the bit stream present much more of a problem. Cluster 
addressing, variable word-length coding, and prc all serve to increase 
the vulnerability of the system to digital transmission errors. A long 
burst of errors would, in most instances, cause picture breakup for 
many seconds, until some updating procedure could restore the 
receiver frame memory to its proper state. 

Although long bursts of digital transmission errors cannot easily 
be corrected, they can, in most cases, be detected fairly easily. The 
receiver could then switch to a frame repeat mode during that portion 
of the picture for which the frame memory is known to be in error, 
thus avoiding the picture breakup associated with free running 
operation. With no movement in the scene, errors would not affect 
picture quality. With movement, however, errors could, for example, 
cause the lower half of the picture to freeze for several seconds until it 
could be updated via pcm. 

Recovery from transmission errors can be speeded up considerably 
if the transmitter can be made aware of their existence and general 
location by feedback from the receiver. The transmitter could then 
simply zero out the offending portion of its frame memory and senda 
control signal telling the receiver to do the same. The erroneous 
portion of the receiver frame memory will then be updated 
automatically. 

Somewhat more complicated schemes can be devised that utilize 
feedback of error status and retransmission of incorrectly received 
data blocks. Some extra buffering is needed (the required amount 
depends on the channel delay), but erroneous data will not enter the 
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receiver frame memory except on rare occasions when the error 
detection algorithm fails. With these techniques, periods of very 
noisy transmission simply cause the transmitter buffer to fill, which 
automatically invokes lower-resolution coding modes or frame repeat- 
ing to match the rate of data generation with the currently available 
channel capacity. 


X. SIMULATION OF THE CODER 


A simulation of the coder was constructed to observe what the 
picture quality would be in an actual system in the absence of digital 
transmission errors. By and large, the simulator performed all the 
coding operations that would significantly affect picture quality; 
however, many shortcuts were taken. 

Synchronization of the camera, Pom coder, and simulator was 
maintained through the same 2-MHz clock; thus, phase-locking and 
stability problems were sidestepped. Peak-signal-to-rms noise ratio 
of the input video signal was above 40 dB; thus, problems of analog 
transmission to the coder were not considered. 

The field delays were obtained using a core memory configured as a 
tapped delay line. Most of the other circuitry was TTL or Mos. A 
buffer was not constructed. Instead, an up-down counter and threshold 
detection logic was used to implement the mode control features 
previously discussed. This approach also made construction of a 
variable word-length coder unnecessary, although presently available 
inexpensive solid-state RoMs make this a fairly easy task. The display 
was obtained by incorporating the field interpolation circuitry of 
Fig. 7 into the simulator. Normally, this logic is required only at the 
recelver. 

For scenes of people engaged in conversation, it was necessary to 
delay the voice signal by about 100 ms to obtain a match with the 
moving lips. Most of this delay is due to the temporal filtering discussed 
previously. The remainder is due to the field delay between input and 
display (see Fig. 7). In fact, a completely satisfying match between 
voice and lips is not obtainable because of the blurring of moving 
areas caused by the temporal filtering. 


XI. CONCLUSION 


In this paper, a frame-to-frame coder for videotelephone signals is 
described that operates at a relatively low bit rate compared with 
previous coders (200 kb/s or 0.1 bit per pel for an original signal of 1 
MHz). The coder was designed on the assumption that faithful 
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rendition of large moving areas in a scene is not essential for effective 
interactive visual communication to take place. Whether or not this 
assumption holds in the majority of situations remains to be seen, 
but it is conceivable that if users are made aware of the considerable 
economic saving involved, they will put up with a certain amount of 
visible distortion in the display. 

Graphics and scenes containing little or no movement are portrayed 
without degradation. Low-key face-to-face conversations contain 
detectable blurring of moving areas, but for many users this may not 
be highly objectionable. However, large moving areas are very visibly 
blurred, sometimes to the point of being nonrecognizable. 

The one-way transmission delay of the coder is comparable to the 
nominally acceptable figure of 300 ms. If the delay of the digital 
transmission channel is also significant compared with this, as it would 
be, for example, on an earth satellite circuit, then interactive communi- 
cation will be severely hampered. Also, special measures must be taken 
to deal with digital transmission errors. The data generated by the 
coder are in highly sensitive form. Thus, if some of them arrive in- 
correctly at the receiver, precautions must be taken to ensure that 
they do not corrupt legitimate information which has already been 
received. 

The techniques described here apply also to higher resolution 
pictures, e.g., 525-line standard broadcast rate signals. Indeed, since 
moving areas do not require any more resolution than with video- 
telephone, the channel bit rate should not be very much higher either. 
Graphics and scenes containing no movement would be displayed 
with much higher resolution. However, the coder itself would also be 
more expensive. 

Much work remains to be done before it will be known if the tech- 
niques described here are useful in providing an acceptable compromise 
between slow-scan facsimile transmission and full rendition of scenes 
containing movement. Coding for redundancy reduction will remain 
practical only if costs of logic and storage fall faster than costs of 
transmission. Also, the requirements of future visual communication 
systems may change drastically after users begin to learn how to use 
them effectively in their day-to-day lives. 
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APPENDIX 


Here, analytical expressions are given for the power transfer P 
versus temporal frequency characteristics of Fig. 3. Let z(t) be the 
light intensity falling on a point of the television camera target, x(t) 
the output signal as that point is read out of the camera, and y(¢) 
the temporally filtered signal. 7’ is the time between normal frames, 


i.e., 1/30 second. 


Fig. 3a— Camera integration only. 


1 t 
Tr ae 2(s)ds 


=f z(s)ds — cf z(s)ds. 


Taking Fourier transforms, 
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Fig. 83b>—Temporal averaging. 
y(t) = ax(t) + gy(¢ — T). 


Taking Fourier transforms, 











Y(w) = 7X (w) + 3Y (we? 
Y(w)|? _ 1 
Bae ~ (5 — 4 cos wT) 
P, = Y(w)|?__-_s- 2(1 — cos wT) 
* |Z(e)| — wT?(5 — 408 wT)’ 





Fig. 3c—Temporal averaging and frame repeating. 


y(t) = 22(f) + ay(t — 27). 








From (7), 
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a Y(w)|? 21 — cos wf) 
_ Ze) ~ oT?(5 — 4 cos 2wT) 
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A Novel Implementation of Digital 
Phase Shifters 
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A novel technique ts presented for implementing a variable digital phase 
shifter which ts capable of realizing noninteger delays. The theory behind 
the technique is based on the idea of first interpolating the signal to a high 
sampling rate, then using an integer delay, and finally decimating the 
signal back to the original sampling rate. Efficient methods for performing 
these processes are discussed in this paper. In particular, it 1s shown that 
the digital phase shifter can be tmplemented by means of a simple con- 
volution at the sampling rate of the original signal. 


I. INTRODUCTION 


In digital systems, linear phase shift or delay of a signal waveform 
by an integer multiple of the sampling period is a simple process that 
can be implemented as a cascade of unit delays in the network. If, 
however, it is desired to delay the signal waveform by an amount not 
equal to an integer multiple of the sampling period, then the process 
is considerably more difficult. In this case, the signal must be interpo- 
lated to obtain new samples of its waveform at noninteger sample 
times. 

In this paper, we propose a novel implementation for achieving 
such noninteger delays. The theory is based on the application of the 
concepts of decimation and interpolation proposed by Schafer and 
Rabiner! and Crochiere and Rabiner? It is shown that the actual 
implementation of the phase shifter or interpolator can be achieved 
by means of a simple convolution. 

Applications in which such noninteger delays in the signal waveform 
are required often occur when digital systems must interface with 
analog systems. For example, in the cancellation of echoes, digital 
systems are often used to generate artificial echoes by means of a 
simulation of an echo model. These artificial echoes are then sub- 
tracted from the original analog signal to cancel its echo. For best 
cancellation, the digital simulated echo may have to be delayed by a 
noninteger multiple of the sampling period. 
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A second potential application occurs when multiple signals must 
be processed together such as in a phased-array antenna system 
(e.g., for seismic processing). In this case, the signal waveforms from 
the various elements must be shifted by noninteger multiples of the 
sampling period relative to each other. 

A third application of noninteger delays is in pitch, synchronous 
synthesis of speech.? In this case, a parametric representation of 
speech is generated at a fixed sampling rate (usually 100 Hz) ; however, 
the synthesis parameters are required at time instances between the 
sampling intervals to avoid producing transients in the synthesized 
signal. Using the variable phase shifter proposed in this paper, the 
synthesis parameters can be readily interpolated to any point between 
sampling intervals. 


Il. BASIC CONCEPTS OF THE PHASE SHIFTER 


Figure 1 illustrates the basic operation of the phase shifter. To 
implement a delay of 1/D samples, where / and D are any integers, the 
sampling rate, f,, of the input signal x(n) is first increased by an 
integer factor D [by inserting D — 1 zero-valued samples between 
each sample of x(n) ]. The resulting signal v(n) is then filtered by a 
low-pass filter h(n) (generally a linear-phase Fir filter is used here) 
to remove its periodic frequency components, which are centered about 
integer multiples of the original sampling frequency.!? The output of 
the filter w(n) is an interpolated version of the input signal x(n). The 
signal u(n) is then delayed by J samples at the high sampling rate to 
produce the signal w(n) = u(n — J). It will be assumed that J satisfies 
the condition. 

02172), (1) 


Finally, the output y(n) is obtained by desampling or decimating 
w(n), 1.e., by choosing every Dth sample of w(n). The net effect is to 
delay the original signal x(n) by a noninteger delay of (J/D)T where 
T = 1/f, is the sampling period at the low rate. In addition, an integer 
delay is introduced in the signal due to the delay of the low-pass 
filter h(n). 

The structure in Fig. 1 can be analyzed in a straightforward manner. 
Let X (e”), V(e*), We”), Y (e*), and H (e”) be the Fourier transforms 
of x(n), v(n), w(n), y(n), and h(n) respectively. Then, the relationships 


x(n) v(n) LPF u(n) win) y(n) 
h(n) 
f; Df, Df, Df, f, 
Fig. 1—Block diagram of the phase shifter. 
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between them can be given as in Refs. 1 and 2: 


V(ew) = X(e*?), (2) 
W (eo) = H(e)e-mV (0H), (3) 
and 
1 P=! : ; 
¥ (6) = 5X W (er emiPeii2), (4) 
D m=0 
In eq. (4), the terms in the summation for m = 1, 2, ---, D — 1 corre- 


spond to high-frequency components of W (e*), which are aliased into 
the low-frequency band from 0 to f,/2 due to the desampling process. 
We assume that the low-pass filter H(e”) attenuates these high- 
frequency components to a point where such aliasing can be considered 
negligible. That is, it has a stop-band cutoff frequency of 1/2D (normal- 
ized to the high sampling rate Df,) and a stop-band ripple 6, that is 
sufficiently small to prevent aliasing. With these assumptions, (4) 
becomes 


Y (e%) & FW (eH?) (5) 
and with the aid of (2) and (8) it can be written as 
Y (¢i4) SH (cielD)e- Ht DY (eed) 

mK H (eID) et PX (eH, (6) 


We now assume that AH (e”) is a FIR filter with exactly linear phase 
and has a unit sample response duration of N samples. Then, its 
delay will be (NV — 1)/2 samples at the high sampling rate. If it is 
desired that this delay be an integer delay at the low sampling rate, 
then N must be chosen such that (NV — 1)/2 is an integer multiple of D. 
That is, 

N-J1 


a = D, (7) 





where J is a positive integer and 
N = 21D +1. (8) 


If the particular application does not require that the delay of the 
filter appear as an integer delay at the low sampling rate, then condition 
(8) is entirely optional and need not be used. 

We now impose the constraint that the passband response of 
H (e#!)) have a gain of D and be essentially flat (i.e., have very small 
passband ripples). Then the filter response of H (e*/”) over the pass- 
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band is approximately [assuming (8) applies | 


H (ei#!D) & Demo ID)UN—1) 121 








= De®", (9) 
Substituting (9) into (6) gives the final desired result : 
Y(e*) tor — jot /D 
X(e*) —° oe — 
or in terms of 2-transforms 
'<C ee —1/D 
Xo= gig. (11) 


Thus, the structure in Fig. 1 is essentially an all-pass network Lover 
the passband of H(e/?) ] with a fixed integer delay of J samples due 
to the processing delay of the low-pass filter h(n) and a variable 
noninteger delay of 1/D samples. If N does not satisfy condition (8), 
then J in eqs. (10) and (11) will not be an integer. In either case, the 
output, y(n), in Fig. 1 is an approximation to x(n — 1/D — I). 


Ill. IMPLEMENTATION OF THE PHASE SHIFTER 


The design of the phase shifter in Fig. 1 suggests a structure which 
involves two different sampling rates. In this section, we show that 
the actual implementation of the phase shifter can be achieved con- 
siderably easier as a straightforward convolution at the low sampling 
rate. 

Since the duration of h(n) is N samples and D — 1 out of every D 
samples of v(n) are zero valued, the filter h(n) spans approximately 
N/D nonzero samples of v(n). More precisely, because of the constraint 
imposed on JN in (8), h(n) spans Q nonzero samples of v(n) for the 
computation of some output points and Q — 1 nonzero samples of 
v(n) for the computation of other output points [Q is defined in 
eq. (13) ]. To avoid this implementation difficulty, it is convenient to 
consider instead a new filter h’(n) whose length N’ is 


N’ = QD 2 N, (12) 


where h’(n) is obtained by extending h(n) with N’ — N zero-valued 
coefficients. Obviously, the filter h’(n) has the same exact frequency 
response and delay as h(n), but it spans exactly Q nonzero samples of 
v(n) [although one nonzero sample of v(m) may be multiplied by a 
zero valued coefficient of h’(n) ]. Since we wish to keep N’ as small 
as possible, consistent with (12) we can choose Q to be 


a=-|5|: (13) 
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where the brackets indicate that the number is rounded to the next 
largest integer. 

With these assumptions, we can now relate the output y(n) in Fig. 1 
to x(n) and h’(n) by the expression? 


y(n) = WIERD + (-1) © Dien — by, (14) 


where @ corresponds to modulo addition. By letting 
glk) =A [kKD+ (-l) @D] k=0,1,:---,Q-1, (15) 
(14) then becomes 


y(n) = ¥ g(B)a(n — by, (16) 


which is the form of a simple convolution. Therefore, the phase shifter 
can be implemented by a Q point convolution of x(n) with g:(n), where 
gi(n) is an appropriate subset of the coefficients of h’(n). To obtain a 
zero incremental phase shift, we use the coefficients {go(0) = h’(0), 
go(1) = h’'(D), ---, g(Q — 1) = h’L(Q — 1)D]}. To obtain a delay 
of (l/D)T (or a phase shift of wl/D), we use the coefficients 
{9:(0) = h’[(—) @ D], m1) = h'LD + (-) @ DI, --+, 9Q-—DN 
= h’'[(Q — 1)D + (—l) ® D]}. If we want a variable phase shifter, 
we can store all D sets of coefficients and use the appropriate set as 
suggested in Fig. 2. 


— SELECTOR FOR 
CHOOSING DESIRED 
—_* 
eo 
e 
e 
e 





\ 





Fig. 2—A practical implementation of a variable phase shifter. 
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V. CONCLUSIONS 


We have presented a method for designing an incremental digital 
phase shifter that can shift the phase of a waveform by a noninteger 
number of samples. Conceptually, the process can be thought of as a 
sample rate increase, a delay, and a sample rate decrease as indicated 
in Fig. 1. Practically, it can be implemented as a straightforward 
convolution as shown in Fig. 2. From the discussion of the theory, 
it is also clear that the design trade-offs of the phase shifter are directly 
related to the characteristics of the low-pass Fir filter. That is, the 
passband ripples of H(e”) determine how close the phase shifter is to 
' an ideal all-pass network (over the passband), and the stop-band 
ripples determine the amount of distortion due to aliasing. Finally, 
the cutoff frequency of the filter determines the usable frequency 
range of the phase shifter. 
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